dev-resources.site
for different kinds of informations.
How can i stop my kafka consumer from consuming messages ?
Published at
7/29/2024
Categories
kafka
springboot
apachekafka
help
Author
shikha_d3b255ec768d55094f
Author
25 person written this
shikha_d3b255ec768d55094f
open
I am using below functions but my kafkaListener is keeps on consuming messages even if my consumer is in paused state.
import org.apache.kafka.clients.consumer.Consumer;
private Consumer<String, String> kafkaConsumer;
public void pauseKafkaConsumer() {
if (!kafkaConsumer.paused().isEmpty()) {
return;
}
// Pause all assigned partitions
Collection<TopicPartition> assignedPartitions =
kafkaConsumer.assignment();
kafkaConsumer.poll(0);
kafkaConsumer.pause(assignedPartitions);
}
// Resume the Kafka consumer
public void resumeKafkaConsumer() {
// Resume all paused partitions
Collection<TopicPartition> pausedPartitions =
kafkaConsumer.paused();
kafkaConsumer.resume(pausedPartitions);
}
@KafkaListener(topics = "#{'${spring.kafka.consumer.topic}'}", groupId = "#{'${spring.kafka.consumer.groupId}'}", containerFactory = "kafkaListenerContainerFactory")
public void consume(String stream, Consumer<?, ?> consumer, Acknowledgment acknowledgment) {
this.kafkaConsumer = (Consumer<String, String>) consumer;
if (getEventCount() > 10) {
pauseKafkaConsumer();
return;
}
}
apachekafka Article's
30 articles in total
Mastering Apache Kafka: A Complete Guide to the Heart of Real-Time Data Streaming
read article
AIM Weekly for 11/11/2024
read article
Apache Kafka: A Simple Guide to Messaging and Streaming
read article
Design a real-time data processing
read article
Building a Scalable Data Pipeline with Apache Kafka
read article
Building a Scalable Data Pipeline with Apache Kafka
read article
Implementing AI with Scikit-Learn and Kafka: A Complete Guide
read article
Understanding the Importance of Kafka in High-Volume Data Environments
read article
How can i stop my kafka consumer from consuming messages ?
currently reading
Getting Started with Apache Kafka: A Beginner's Guide to Distributed Event Streaming
read article
π Apache Kafka Cluster Explained: Core Concepts and Architectures π
read article
WarpStream Newsletter #5: Dealing with Rejection, Schema Validation, and Time Lag
read article
Dealing with rejection (in distributed systems)
read article
Apache Kafka on Amazon Linux EC2
read article
Announcing WarpStream Schema Validation
read article
The Kafka Metric Youβre Not Using: Stop Counting Messages, Start Measuring Time
read article
WarpStream Newsletter #4: Data Pipelines, Zero Disks, BYOC and More
read article
Integrating Apache Kafka with Apache AGE for Real-Time Graph Processing
read article
Integrating Apache Kafka with Apache AGE for Real-Time Graph Processing
read article
Multiple Regions, Single Pane of Glass
read article
FLaNK-AIM: 20 May 2024 Weekly
read article
Secure by default: How WarpStreamβs BYOC deployment model secures the most sensitive workloads
read article
Zero Disks is Better (for Kafka)
read article
FLaNK AI-April 22,Β 2024
read article
Pixel Federation Powers Mobile Analytics Platform with WarpStream, saves 83% over MSK
read article
FLaNK AI - 15 April 2024
read article
WarpStream Newsletter #3: Always Be Shipping
read article
Introducing WarpStream Managed Data Pipelines for BYOC Clusters
read article
Apache Kafka
read article
FLaNK-AIM Weekly 06 May 2024
read article
Featured ones: