Commit Graph

  • 9a8af1499c Fix KafkaClient->SimpleClient references 0.9 Dana Powers 2016-01-07 17:57:24 -08:00
  • d4e85ecd1d Update docs for release w/ new async classes Dana Powers 2016-01-07 17:14:49 -08:00
  • 2a2e77aa1e Drop sphinxcontrib-napoleon in favor of sphinx.ext.napolean Dana Powers 2016-01-07 17:10:04 -08:00
  • 19d403c743 Add metadata_max_age_ms and retry_backoff_ms options to async client Dana Powers 2016-01-07 17:08:32 -08:00
  • e080c6b0cd Docstring updates Dana Powers 2016-01-07 17:03:08 -08:00
  • c8deb0c276 Reorg kafka imports Dana Powers 2016-01-07 16:58:11 -08:00
  • 31c3d59ee3 Reduce partitions for offset commit topic in 0.9 broker configs Dana Powers 2016-01-04 11:33:08 -08:00
  • 16e35c9d16 Drop request_timeout_ms override in consumer group test Dana Powers 2016-01-04 00:55:55 -08:00
  • 1a0b86625f Dont use consumer_timeout_ms in simple kafka_consumer test Dana Powers 2016-01-04 00:19:38 -08:00
  • 16c834119b Fix future redefine bug in client.poll Dana Powers 2016-01-04 00:18:58 -08:00
  • bb433e27f0 Call errbacks with future.exception Dana Powers 2016-01-03 23:31:09 -08:00
  • 54d758e813 Dont pylint in pypy tests - it seems to take forever and doesn't add anything Dana Powers 2016-01-03 22:47:15 -08:00
  • c8226d030a Dont use consumer_timeout_ms in kafka blocking test Dana Powers 2016-01-03 22:46:18 -08:00
  • 09358f9c82 Fast heartbeats during consumer group tests Dana Powers 2016-01-03 22:43:39 -08:00
  • 03d37dff5c Import queue from six.moves Dana Powers 2016-01-03 19:50:21 -08:00
  • ed8f2212bf Attempt to fix travis shell magic for pypy Dana Powers 2016-01-03 18:49:16 -08:00
  • 42ea4f4913 Catch py3 ConnectionErrors Dana Powers 2016-01-03 18:45:32 -08:00
  • 1bcb9f029d Message value can be None Dana Powers 2016-01-03 18:17:23 -08:00
  • b13c92306c Dont run pylint on python2.6 (pylint no longer supports) Dana Powers 2016-01-03 18:09:13 -08:00
  • 8d85ca2d44 Dont run travis tests against 0.8.1 (keep 0.8.1.1); remove UNIT_AND_LINT_ONLY Dana Powers 2016-01-03 18:01:30 -08:00
  • ece72e4745 Move pytest options to [pytest] tox section Dana Powers 2016-01-03 18:00:45 -08:00
  • 95e7a65a8e Override Message __hash__ to use _encode_self and not recalc crcs Dana Powers 2016-01-03 18:00:03 -08:00
  • ca88921ee7 Fix TRAVIS_PYTHON_VERSION magic in .travis.yml Dana Powers 2016-01-03 17:29:23 -08:00
  • 520d3c1484 Add pytests for KafkaConsumer group assignments Dana Powers 2016-01-03 17:21:58 -08:00
  • 8f15262c79 Assert Message value and (optional) key are bytes Dana Powers 2016-01-03 17:12:15 -08:00
  • d73f26d57f Add __hash__ method to Struct Dana Powers 2016-01-03 17:11:46 -08:00
  • 6d48a1cc5f Use parameterized pytests in test_client_async; add pytest-mocker plugin Dana Powers 2016-01-03 16:28:49 -08:00
  • fa7ecdaf3f Use 2-second KafkaClient timeout in failover tests Dana Powers 2016-01-03 16:25:12 -08:00
  • 50f6a25ceb Update consumer integration tests to use new (group) KafkaConsumer Dana Powers 2016-01-03 16:18:08 -08:00
  • 9acb689015 bootstrap_servers no longer required in KafkaConsumer (localhost default) Dana Powers 2016-01-03 16:14:52 -08:00
  • 0334fee550 Switch to new KafkaConsumer in module imports Dana Powers 2016-01-03 16:14:37 -08:00
  • df75751238 Add deprecated methods to KafkaConsumer w/ notes on alternatives Dana Powers 2016-01-03 16:12:22 -08:00
  • abdbc6ca27 Drop unused method from ConsumerCoordinator Dana Powers 2016-01-03 16:10:40 -08:00
  • 71af307f4e Disable pylint error on __iter__ (next is provided via six.Iterator) Dana Powers 2016-01-03 16:09:52 -08:00
  • 16c56235d2 Keep support for old largest/smallest auto_offset_reset options Dana Powers 2016-01-03 16:08:56 -08:00
  • 2b43ae1609 Return empty dict from KafkaConsumer.poll if no records Dana Powers 2016-01-03 16:08:11 -08:00
  • 79aa0f0489 Support consumer_timeout_ms in new KafkaConsumer Dana Powers 2016-01-03 16:07:24 -08:00
  • 9b07bfb529 Check for 0.8.2 GroupCoordinator quirk in BrokerConnection Dana Powers 2016-01-03 16:06:35 -08:00
  • 5c45ec13f3 Check api_version in ConsumerCoordinator Dana Powers 2016-01-03 16:03:30 -08:00
  • fae1a227b1 Add api_version config to KafkaConsumer; disable features inline by version Dana Powers 2016-01-03 16:00:16 -08:00
  • 30fefa9b4f Add KafkaClient.check_version() to guess broker version Dana Powers 2016-01-03 15:46:37 -08:00
  • 4dc1fbab30 Fix bug in _initiate_connect preventing reconnect to 'bootstrap' Dana Powers 2016-01-03 15:45:18 -08:00
  • 995ea64b8f Fix timeout bug in BrokerConnection.connect() Dana Powers 2016-01-03 15:43:43 -08:00
  • 2b289224f9 Raise exception in KafkaConsumer on unrecognized kwargs Dana Powers 2016-01-02 13:22:33 -08:00
  • 00a1e6e819 Batched message methods now return dict of lists Dana Powers 2016-01-02 13:21:16 -08:00
  • 24a4c2a7c5 Improve iterator interface Dana Powers 2016-01-02 13:18:10 -08:00
  • 976970f89a Fixup b6a2ad9: Fail with ConnectionErrors in BrokerConnection.send Dana Powers 2016-01-02 13:12:29 -08:00
  • 77350fe0ca Add default sphinx docs for kafka.coordinator and kafka.protocol Dana Powers 2016-01-01 23:24:43 -08:00
  • b6e9b7f74d Disable pylint errors for py2/py3 compatibility workarounds Dana Powers 2016-01-01 22:59:55 -08:00
  • eab5064929 Use log exception for metadata failure in async producer Dana Powers 2016-01-01 22:57:08 -08:00
  • cb1b52f705 Use python3 compatible next() in roundrobin assignor Dana Powers 2016-01-01 22:49:34 -08:00
  • b957de1fd9 sys.maxint not supported in python3 Dana Powers 2016-01-01 22:48:24 -08:00
  • 1a6ff2615c Use pytest instead of nose for running tests via tox Dana Powers 2016-01-01 15:04:51 -08:00
  • cb325af5ff Use find_packages() for setup.py Dana Powers 2016-01-01 15:03:36 -08:00
  • b1e0aef468 Skeleton tests for async kafka client Dana Powers 2016-01-01 12:11:32 -08:00
  • 9fe904e9bb Struct __eq__ Dana Powers 2016-01-01 12:11:08 -08:00
  • 9258bb4ba5 Remove some unused bits from kafka.consumer.group Dana Powers 2015-12-31 11:21:44 -08:00
  • 89e22a0e45 Improve request pipelining in consumer iterator Dana Powers 2015-12-31 11:21:16 -08:00
  • 61ccbc5f7b Fixup heartbeat config assertion Dana Powers 2015-12-31 11:20:01 -08:00
  • bc2688e69c Support message decompression Dana Powers 2015-12-31 11:18:49 -08:00
  • 650b134f0b Towards version 1.0 Dana Powers 2015-12-30 16:28:49 -08:00
  • 14de82535a Support simple message iteration in Fetcher and new KafkaConsumer Dana Powers 2015-12-30 16:26:46 -08:00
  • 93b8afed01 Rename AbstractCoordinator -> BaseCoordinator, log as kafka.coordinator Dana Powers 2015-12-30 16:17:25 -08:00
  • 422050f952 Prefer assert or more-specific error to IllegalState / IllegalArgument Dana Powers 2015-12-30 16:16:13 -08:00
  • 59c0513148 Catch GroupCoordinatorNotAvailableError in GroupCoordinatorResponse handler Dana Powers 2015-12-30 13:17:16 -08:00
  • 71e7568fcb Start test fixtures in new session to avoid propagating signals (i.e., Ctrl-C) simple Dana Powers 2015-12-30 13:05:11 -08:00
  • 61c6e18408 Start test fixtures in new session to avoid propagating signals (i.e., Ctrl-C) Dana Powers 2015-12-30 13:05:11 -08:00
  • e093ffefae More Docstring Improvements Dana Powers 2015-12-30 12:51:34 -08:00
  • 1dd9e8bb05 Fix blacked_out typo in least_loaded_node Dana Powers 2015-12-30 12:46:42 -08:00
  • b6a2ad9caa BrokerConnection: wrap socket errors in retriable ConnectionError Dana Powers 2015-12-30 12:45:24 -08:00
  • 86c89cdaff Rename KafkaClient.connection_failed -> is_disconnected Dana Powers 2015-12-30 12:23:42 -08:00
  • 8dcfa96542 Drop unused KafkaConsumer._ensure_not_closed method Dana Powers 2015-12-30 12:21:50 -08:00
  • f161cd11aa Make group_id a Coordinator kwarg (not arg) for consistency Dana Powers 2015-12-30 12:21:00 -08:00
  • 61161d8335 Improve removed tasks handing in DelayedTaskQueue.pop_ready Dana Powers 2015-12-30 12:18:02 -08:00
  • cfae9e3fa3 Remove unnecessary calls in KafkaClient._poll Dana Powers 2015-12-30 11:50:42 -08:00
  • 9bc01657ed Resolve delayed task futures in KafkaClient.poll Dana Powers 2015-12-30 11:49:22 -08:00
  • 88cf1b5e45 Remove unused connection_delay method from KafkaClient Dana Powers 2015-12-30 11:48:08 -08:00
  • 3afdd285a3 Switch configs from attributes to dict to make passing / inspecting easier Dana Powers 2015-12-29 19:08:35 -08:00
  • e5c7d81e7c Use _next_correlation_id() method to avoid int overflows Dana Powers 2015-12-29 19:07:44 -08:00
  • 8f0d1c1716 Fix log statement bug (via pylint) Dana Powers 2015-12-29 19:06:04 -08:00
  • a766495355 Clean more ConsumerCoordinator docstrings Dana Powers 2015-12-29 19:05:29 -08:00
  • 3748bade28 Catch and retry on RequestTimedOutError in producer failover test Dana Powers 2015-12-29 17:49:04 -08:00
  • c549a2b403 Use ConnectionError rather than new DisconnectError Dana Powers 2015-12-29 17:48:10 -08:00
  • 563e38a84f Dont warn in AutoCommitTask.disable if not previously scheduled Dana Powers 2015-12-29 17:36:11 -08:00
  • 6e20e0bb52 Improve OffsetCommit error logging Dana Powers 2015-12-29 17:34:17 -08:00
  • 39e7562b31 Log request failures in AbstractCoordinator._failed_request Dana Powers 2015-12-29 17:27:36 -08:00
  • 8811326443 _handle_group_coordinator_response should resolve future with the coordinator id Dana Powers 2015-12-29 17:21:25 -08:00
  • f82eb8cdf6 Improve heartbeat logging Dana Powers 2015-12-29 17:19:51 -08:00
  • ab1578444c Log as INFO: group join, leader election, partition assignments Dana Powers 2015-12-29 17:04:24 -08:00
  • 2a2733d4fc Improve various docstrings Dana Powers 2015-12-29 17:01:41 -08:00
  • b7d1ed3fb4 Remove extra raise and old TODO comment in coordinators Dana Powers 2015-12-29 16:59:59 -08:00
  • aa35c4e424 Make add_assigned_partition private(ish) in SubscriptionState Dana Powers 2015-12-29 16:58:38 -08:00
  • 357073cc18 Make more methods private(ish) in AbstractCoordinator Dana Powers 2015-12-29 16:58:08 -08:00
  • 6dea93dfdb Remove recursive client.poll call in ConsumerCoordinator Dana Powers 2015-12-29 15:46:50 -08:00
  • ac57ada2bd Pass auto commit parameters from KafkaConsumer to ConsumerCoordinator Dana Powers 2015-12-29 15:45:48 -08:00
  • ab1e1d014f Fix broken SubscriptionState.is_assigned call Dana Powers 2015-12-29 15:44:39 -08:00
  • 80078c2ed4 Fix HeartbeatTask request_in_flight handling Dana Powers 2015-12-29 15:43:27 -08:00
  • 70d2f2630d kafka.consumer.group.KafkaConsumer: 0.9 coordinator-aware consumer - use ConsumerCoordinator to manage group membership - dynamically assign partitions via group using topic subscription - use upstream configuration parameters as __init__ kwargs - metrics and SSL are still TODO Dana Powers 2015-12-28 23:10:00 -08:00
  • 161d9ffcf8 ConsumerCoordinator (based on upstream Java client) Dana Powers 2015-12-28 15:16:55 -08:00
  • baab076c7e AbstractCoordinator (modelled after upstream Java class) Dana Powers 2015-12-28 15:05:08 -08:00