openstack/deb-python-cassandra-driver
Code Issues Proposed changes

Minor doc clarification about concurrency limits

Browse Source
This commit is contained in:
Tyler Hobbs
2014-12-03 16:10:05 -06:00
parent 92439a1c0e
commit b0bb243db5
2 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
cassandra/concurrent.py
View File

@@ -41,11 +41,13 @@ def execute_concurrent(session, statements_and_parameters, concurrency=100, rais
raised. raised.
The `concurrency` parameter controls how many statements will be executed The `concurrency` parameter controls how many statements will be executed
concurrently. It is recommended that this be kept below the number of concurrently. When :attr:`.Cluster.protocol_version` is set to 1 or 2,
it is recommended that this be kept below 100 times the number of
core connections per host times the number of connected hosts (see core connections per host times the number of connected hosts (see
:meth:`.Cluster.set_core_connections_per_host`). If that amount is exceeded, :meth:`.Cluster.set_core_connections_per_host`). If that amount is exceeded,
the event loop thread may attempt to block on new connection creation, the event loop thread may attempt to block on new connection creation,
substantially impacting throughput. substantially impacting throughput. If :attr:`~.Cluster.protocol_version`
is 3 or higher, you can safely experiment with higher levels of concurrency.
Example usage:: Example usage::

5
docs/performance.rst
View File

@@ -242,7 +242,10 @@ dramatically:
~/python-driver $ python benchmarks/callback_full_pipeline.py -n 100000 --hosts=127.0.0.1,127.0.0.2,127.0.0.3 --libev-only --threads=1 ~/python-driver $ python benchmarks/callback_full_pipeline.py -n 100000 --hosts=127.0.0.1,127.0.0.2,127.0.0.3 --libev-only --threads=1
Average throughput: 679.61/sec Average throughput: 679.61/sec
Until this is improved, you should limit the number of callback chains you run. When :attr:`.Cluster.protocol_version` is set to 1 or 2, you should limit the
number of callback chains you run to rougly 100 per node in the cluster.
When :attr:`~.Cluster.protocol_version` is 3 or higher, you can safely experiment
with higher numbers of callback chains.
For many use cases, you don't need to implement this pattern yourself. You can For many use cases, you don't need to implement this pattern yourself. You can
simply use :meth:`cassandra.concurrent.execute_concurrent` and simply use :meth:`cassandra.concurrent.execute_concurrent` and