Metadata-Version: 2.1
Name: django-kafka-streamer
Version: 1.4.1
Summary: Stream data to Apache Kafka.
Author-email: Kostiantyn Korikov <lostclus@gmail.com>
Project-URL: Repository, http://github.com/lostclus/django-kafka-streamer
Project-URL: Documentation, http://django-kafka-streamer.readthedocs.io/
Keywords: Django,Kafka,microservices
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Framework :: Django
Classifier: Framework :: Django :: 3
Classifier: Framework :: Django :: 3.2
Classifier: Framework :: Django :: 4
Classifier: Framework :: Django :: 4.0
Classifier: Framework :: Django :: 4.1
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: Django>=3.2
Requires-Dist: kafka-python-ng>=2.2.2
Requires-Dist: celery>=5.1.0
Provides-Extra: tests
Requires-Dist: pytest; extra == "tests"
Requires-Dist: pytest-cov; extra == "tests"
Requires-Dist: pytest-djangoapp; extra == "tests"
Provides-Extra: docs
Requires-Dist: sphinx; extra == "docs"
Requires-Dist: sphinx_rtd_theme; extra == "docs"

django-kafka-streamer
=====================

.. image:: https://github.com/lostclus/django-kafka-streamer/actions/workflows/tests.yml/badge.svg
    :target: https://github.com/lostclus/django-kafka-streamer/actions

.. image:: https://readthedocs.org/projects/django-kafka-streamer/badge/?version=latest
    :target: http://django-kafka-streamer.readthedocs.io/
    :alt: Documentation

.. image:: https://img.shields.io/pypi/v/django-kafka-streamer.svg
    :target: https://pypi.org/project/django-kafka-streamer/
    :alt: Current version on PyPi

.. image:: https://img.shields.io/pypi/pyversions/django-kafka-streamer
    :alt: PyPI - Python Version

.. image:: https://img.shields.io/pypi/djversions/django-kafka-streamer
    :alt: PyPI - Django Version

.. image:: https://img.shields.io/badge/Published%20on-Django%20Packages-0c3c26
    :target: https://djangopackages.org/packages/p/django-kafka-streamer/
    :alt: Published on Django Packages

django-kafka-streamer is a Django application and library for streaming data to
Apache Kafka.

Features:

* Setup signal handlers to ORM models to transparently send create/update/delete
  events to Kafka
* Handle database object relations
* Celery task to stream large amount of data in background

Links:

* GitHub: https://github.com/lostclus/django-kafka-streamer/
* PyPI: https://pypi.org/project/django-kafka-streamer/
* Documentation: http://django-kafka-streamer.readthedocs.io/
* Consumer library: https://github.com/lostclus/aiosafeconsumer
* Example application: https://github.com/lostclus/WeatherApp

Usage:

`yourapp/models.py`::

    from django.db import models

    class MyModel(models.Model):
        field1 = models.IntegerField()
        field2 = models.CharField(max_length=10)

`yourapp/stramers.py`::

    from kafkastreamer import Streamer, register
    from .models import MyModel

    @register(MyModel)
    class MyModelStreamer(Streamer):
        topic = "model-a"

`yourproject/settings.py`::

    INSTALLED_APPS = [
        ...
        "kafkastreamer",
    ]

    KAFKA_STREAMER = {
        "BOOTSTRAP_SERVERS": ["localhost:9092"],
    },

Any changes in ``MyModel`` data will be automatically streamed to Kafka. To
force stream all data in all registered models type::

    python manage.py kafkastreamer_refresh

The data streamed to the ``model-a`` Kafka topic has following structure::

    {
        "_time": "2023-01-01T00:00:00Z",
        "_type": "create",
        "id": 1,
        "field1": 1,
        "field2": "abc"
    }
