Update README from new docs
This commit is contained in:
Dana Powers
97
README.rst
97
README.rst
@@ -1,59 +1,72 @@
|
|||||||
Kafka Python client
|
Kafka Python client
|
||||||
------------------------
|
------------------------
|
||||||
.. image:: https://api.travis-ci.org/dpkp/kafka-python.png?branch=master
|
|
||||||
|
.. image:: https://img.shields.io/badge/kafka-0.9%2C%200.8.2%2C%200.8.1%2C%200.8-brightgreen.svg
|
||||||
|
:target: https://kafka-python.readthedocs.org/compatibility.html
|
||||||
|
.. image:: https://img.shields.io/pypi/pyversions/kafka-python.svg
|
||||||
|
:target: https://pypi.python.org/pypi/kafka-python
|
||||||
|
.. image:: https://coveralls.io/repos/dpkp/kafka-python/badge.svg?branch=master&service=github
|
||||||
|
:target: https://coveralls.io/github/dpkp/kafka-python?branch=master
|
||||||
|
.. image:: https://travis-ci.org/dpkp/kafka-python.svg?branch=master
|
||||||
:target: https://travis-ci.org/dpkp/kafka-python
|
:target: https://travis-ci.org/dpkp/kafka-python
|
||||||
:alt: Build Status
|
.. image:: https://img.shields.io/badge/license-Apache%202-blue.svg
|
||||||
|
:target: https://github.com/dpkp/kafka-python/blob/master/LICENSE
|
||||||
|
|
||||||
.. image:: https://coveralls.io/repos/dpkp/kafka-python/badge.svg?branch=master
|
>>> pip install kafka-python
|
||||||
:target: https://coveralls.io/r/dpkp/kafka-python?branch=master
|
|
||||||
:alt: Coverage Status
|
|
||||||
|
|
||||||
.. image:: https://readthedocs.org/projects/kafka-python/badge/?version=latest
|
kafka-python is a client for the Apache Kafka distributed stream processing
|
||||||
:target: http://kafka-python.readthedocs.org/en/latest/
|
system. It is designed to function much like the official java client, with a
|
||||||
:alt: Full documentation available on ReadTheDocs
|
sprinkling of pythonic interfaces (e.g., iterators).
|
||||||
|
|
||||||
This module provides low-level protocol support for Apache Kafka as well as
|
|
||||||
high-level consumer and producer classes. Request batching is supported by the
|
|
||||||
protocol as well as broker-aware request routing. Gzip and Snappy compression
|
|
||||||
is also supported for message sets.
|
|
||||||
|
|
||||||
Coordinated Consumer Group support is under development - see Issue #38.
|
KafkaConsumer
|
||||||
|
*************
|
||||||
|
|
||||||
Full documentation available on `Read the Docs <https://kafka-python.readthedocs.org/en/latest/>`_
|
>>> from kafka import KafkaConsumer
|
||||||
|
>>> consumer = KafkaConsumer('my_favorite_topic')
|
||||||
|
>>> for msg in consumer:
|
||||||
|
... print (msg)
|
||||||
|
|
||||||
On Freenode IRC at #kafka-python, as well as #apache-kafka
|
KafkaConsumer is a full-featured,
|
||||||
|
high-level message consumer class that is similar in design and function to the
|
||||||
|
new 0.9 java consumer. Most configuration parameters defined by the official
|
||||||
|
java client are supported as optional kwargs, with generally similar behavior.
|
||||||
|
Gzip and Snappy compressed messages are supported transparently.
|
||||||
|
|
||||||
For general discussion of kafka-client design and implementation (not python specific),
|
In addition to the standard KafkaConsumer.poll() interface (which returns
|
||||||
see https://groups.google.com/forum/#!forum/kafka-clients
|
micro-batches of messages, grouped by topic-partition), kafka-python supports
|
||||||
|
single-message iteration, yielding ConsumerRecord namedtuples, which include
|
||||||
|
the topic, partition, offset, key, and value of each message.
|
||||||
|
|
||||||
For information about Apache Kafka generally, see https://kafka.apache.org/
|
By default, KafkaConsumer will attempt to auto-commit
|
||||||
|
message offsets every 5 seconds. When used with 0.9 kafka brokers,
|
||||||
|
KafkaConsumer will dynamically assign partitions using
|
||||||
|
the kafka GroupCoordinator APIs and a RoundRobinPartitionAssignor
|
||||||
|
partitioning strategy, enabling relatively straightforward parallel consumption
|
||||||
|
patterns. See `ReadTheDocs <http://kafka-python.readthedocs.org/master/>`_
|
||||||
|
for examples.
|
||||||
|
|
||||||
License
|
|
||||||
----------
|
|
||||||
Apache License, v2.0. See `LICENSE <https://github.com/dpkp/kafka-python/blob/master/LICENSE>`_
|
|
||||||
Copyright 2015, David Arthur, Dana Powers, and Contributors
|
|
||||||
(See `AUTHORS <https://github.com/dpkp/kafka-python/blob/master/AUTHORS.md>`_)
|
|
||||||
|
|
||||||
Status
|
KafkaProducer
|
||||||
----------
|
*************
|
||||||
The current stable version of this package is
|
|
||||||
`0.9.5 <https://github.com/dpkp/kafka-python/releases/tag/v0.9.5>`_
|
|
||||||
and is compatible with:
|
|
||||||
|
|
||||||
Kafka broker versions
|
<`in progress - see SimpleProducer for legacy producer implementation`>
|
||||||
|
|
||||||
- 0.9.0.0
|
|
||||||
- 0.8.2.2
|
|
||||||
- 0.8.2.1
|
|
||||||
- 0.8.1.1
|
|
||||||
- 0.8.1
|
|
||||||
- 0.8.0
|
|
||||||
|
|
||||||
Python versions
|
Protocol
|
||||||
|
********
|
||||||
|
|
||||||
- 3.5 (tested on 3.5.0)
|
A secondary goal of kafka-python is to provide an easy-to-use protocol layer
|
||||||
- 3.4 (tested on 3.4.2)
|
for interacting with kafka brokers via the python repl. This is useful for
|
||||||
- 3.3 (tested on 3.3.5)
|
testing, probing, and general experimentation. The protocol support is
|
||||||
- 2.7 (tested on 2.7.9)
|
leveraged to enable a KafkaClient.check_version() method that
|
||||||
- 2.6 (tested on 2.6.9)
|
probes a kafka broker and attempts to identify which version it is running
|
||||||
- pypy (tested on pypy 2.5.0 / python 2.7.8)
|
(0.8.0 to 0.9).
|
||||||
|
|
||||||
|
|
||||||
|
Low-level
|
||||||
|
*********
|
||||||
|
|
||||||
|
Legacy support is maintained for low-level consumer and producer classes,
|
||||||
|
SimpleConsumer and SimpleProducer. See
|
||||||
|
`ReadTheDocs <http://kafka-python.readthedocs.org/master/>`_ for API details.
|
||||||
|
|||||||
Reference in New Issue
Block a user