Question 1
A Kafka cluster with three brokers has a topic with 10 partitions and a replication factor set to three. Each partition stores 25 GB data per day and data retention is set to 24 hours.
How much storage will be consumed by the topic on each broker?
10 partitions 25 GB/day = 250 GB total per day for the topic (primary data).
With a replication factor of 3, there are 3 full copies of the data: 250 GB 3 = 750 GB total across the entire cluster.
The cluster has 3 brokers, and Kafka tries to distribute replicas evenly among them: 750 GB 3 brokers = 250 GB per broker on average.
However, due to replication, some partitions have leaders and followers, so there's some overlap and not-perfect distribution. Each broker stores approximately 2/3 of the total topic data (since each broker holds replicas for around 2/3 of the partitions).
2/3 750 GB = 500 GB, but this is shared, so each broker ends up storing ~300 GB of replicated data, including its share of leaders and followers.