Processing Data: Integrating Kafka with Python & Using Consumer Groups

Producers and consumers are applications that write events to and read events from Kafka. In this course, you will focus on integrating Python applications with a Kafka environment, implementing consumer groups, and tweaking Kafka configurations.Begin by connecting to Kafka from Python. You will produce to and consume messages from a Kafka topic using Python. Next, discover how to tweak Kafka broker configurations. You will place limits on the size of messages and disable deletion of topics. Then, publish messages to partitioned topics and explore the use of partitioning algorithms to determine the placement of messages on partitions. Explore consumer groups, which allow a set of consumers to process messages published to partitioned Kafka topics in parallel – without any duplication of effort. Finally, learn different ways to optimize Kafkas performance, using configurations for brokers and topics, as well as producer and consumer apps.