openstack/deb-python-cassandra-driver
Code Issues Proposed changes

Minor doc clarification about concurrency limits

Browse Source
This commit is contained in:
Tyler Hobbs
2014-12-03 16:10:05 -06:00
parent 92439a1c0e
commit b0bb243db5
2 changed files with 8 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
cassandra/concurrent.py
View File

@@ -41,11 +41,13 @@ def execute_concurrent(session, statements_and_parameters, concurrency=100, rais
raised.
The `concurrency` parameter controls how many statements will be executed
concurrently. It is recommended that this be kept below the number of
concurrently. When :attr:`.Cluster.protocol_version` is set to 1 or 2,
it is recommended that this be kept below 100 times the number of
core connections per host times the number of connected hosts (see
:meth:`.Cluster.set_core_connections_per_host`). If that amount is exceeded,
the event loop thread may attempt to block on new connection creation,
substantially impacting throughput.
substantially impacting throughput. If :attr:`~.Cluster.protocol_version`
is 3 or higher, you can safely experiment with higher levels of concurrency.
Example usage::

5
docs/performance.rst
View File

@@ -242,7 +242,10 @@ dramatically:
~/python-driver $ python benchmarks/callback_full_pipeline.py -n 100000 --hosts=127.0.0.1,127.0.0.2,127.0.0.3 --libev-only --threads=1
Average throughput: 679.61/sec
Until this is improved, you should limit the number of callback chains you run.
When :attr:`.Cluster.protocol_version` is set to 1 or 2, you should limit the
number of callback chains you run to rougly 100 per node in the cluster.
When :attr:`~.Cluster.protocol_version` is 3 or higher, you can safely experiment
with higher numbers of callback chains.
For many use cases, you don't need to implement this pattern yourself. You can
simply use :meth:`cassandra.concurrent.execute_concurrent` and