db5ecf69c8
It turned out that distribution of WSGI requests between forks is not even when eventlet is used, which can cause pool timeout issues under load, while there are idle workers with open DB connections, which should have served these HTTP requests. The default max_overflow value should be increased to allow DB oriented services (mostly APIs) handle spikes in number of concurrent requests trying to use a DB. At the same time, the default number of greenlets is decreased: I65b40b9906b75146a0085bbe168f1e6bcae82f21 which effectively causes a particular worker to stop accepting new requests it has no resources (DB connections) to handle and allow other forks accept()'ing on the same FD to proceed. Testing was performed using this script: https://gist.github.com/zzzeek/c69138fd0d0b3e553a1f With 100 greenthreads in the pool, 50 DB connections allowed overflow no pool timeout issues were seen with up to 500 concurrent requests done by ab, while current default values (1000/10) could not handle even 100 concurrent requests. See this ML thread for details: http://lists.openstack.org/pipermail/openstack-dev/2015-December/082717.html DocImpact This change potentially increases the number of connections open to the RDBMS server. If you start seeing "too many connections" errors, please check these settings and adjust them appropriately: https://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_max_connections http://www.postgresql.org/docs/current/static/runtime-config-connection.html#GUC-MAX-CONNECTIONS Closes-Bug: #1535375 Change-Id: I2e9c2a71d8231e0dfbefc6293ad319e1e459beec
26 lines
1005 B
YAML
26 lines
1005 B
YAML
---
|
|
upgrade:
|
|
- |
|
|
The default value of ``max_overflow`` config option
|
|
has been increased from 10 to 50 in order to allow
|
|
OpenStack services heavily using DBs to better handle
|
|
spikes of concurrent requests and lower the probability
|
|
of getting a pool timeout issue.
|
|
|
|
This change potentially leads to increasing of the number
|
|
of open connections to an RDBMS server. Depending on the
|
|
configuration, you may see "too many connections" errors
|
|
in logs of OpenStack services / RDBMS server. The max limit of
|
|
connections can be set by the means of these config options:
|
|
|
|
http://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_max_connections
|
|
http://www.postgresql.org/docs/current/static/runtime-config-connection.html#GUC-MAX-CONNECTIONS
|
|
|
|
For details, please see the following LP:
|
|
|
|
https://bugs.launchpad.net/oslo.db/+bug/1535375
|
|
|
|
and the ML thread:
|
|
|
|
http://lists.openstack.org/pipermail/openstack-dev/2015-December/082717.html
|