Apache Kafka on Confluent Cloud enables organizations to perform real-time data streaming and processing, integrating user data, and managing large transaction volumes efficiently.
| Title | Rating | Mindshare | Recommending | |
|---|---|---|---|---|
| Security Onion | 3.8 | 1.3% | 66% | 3 interviewsAdd to research |
| Cloud Security Connector for Zscaler | 4.5 | 0.8% | 100% | 2 interviewsAdd to research |
Organizations leverage Apache Kafka on Confluent Cloud for several applications such as capturing change data, handling IoT workloads, and fostering data migration and microservices. It is instrumental in publishing and managing data across multiple platforms, ensuring both high throughput and seamless log management. Confluent Cloud's features like scalability, efficient transaction management, and cost-effectiveness, combined with seamless integration with multiple data tools, make it a preferred choice for real-time data processing tasks.
What are the key features?
What benefits and ROI should you look for?
In specific industries, such as finance, real-time data processing is crucial for handling high volumes of transactions and location tracking. In retail, it integrates user data and manages inventory systems efficiently. IoT applications benefit from Apache Kafka on Confluent Cloud by capturing and processing massive data streams from connected devices in real-time.
It's basically four bands of use cases, where we publish data on Kafka topics and stream it across microservices.
Some of the retention policies of Kafka topics have been most beneficial for data management specifically.
Based on our experience, there are different use cases where data needs to be handled in different ways. Sometimes we want to get rid of it once it has been consumed, or we have to store it for a longer period.
Kafka provides handy properties that allow us to directly configure the data, whether to keep it or discard it after use.
I feel the streaming speed, the way messages are processed, and some of the topic features like partitions and offset management are quite handy.
There's one thing that's a common use case, but I don't know why it's not covered in Kafka. When a message comes in, and another message with the same key arrives, the first version should be deleted automatically.
We want to keep only one instance of a message at any given time, the latest one. However, Kafka doesn't have this functionality built-in. It keeps all the data, and we have to manually delete the older versions.
So, I would like to have only one instance of messages, based on the keys. If the key is the same, there should always be the latest message present instead of all versions of that message.
I have been using it for three years. I use the latest version. I work with v3.6.
I would rate the stability a six out of ten. When it's good, it works fine. But as soon as traffic increases and the number of topics on the cluster exceeds a certain limit, it becomes unresponsive.
We then have to get rid of Kafka topics, but even that's not easy because the whole site becomes unresponsive. We don't have easy dashboard access to remove unnecessary topics. That's one issue.
I would give it an eight out of ten for scalability. It's scalable, but there's room for improvement in reliability. When we scale up at the pod level, reliability goes down due to mismanagement of offsets, leading to data loss. Then, there is mismanagement when we scale it up, and then there is a point where we want to scale down because traffic is less.
During scale-down, we also often see data loss. They can work on improving this.
We currently have large enterprise business as our customers.
I would rate my experience with the initial setup of this product, a seven out of ten where one is difficult and ten is easy.
I have not faced any difficulties or challenges while setting this product up. They have proper documentation, so it's easy to go through it and set things up.
It's the cloud solution, so it's deployed on the cloud in our customers' organizations. And they use Confluent Cloud.
It's taken care of by different teams.
Overall, I would rate it an eight out of ten.
I would recommend it because everything is well-documented and straightforward.
We can install Kafka directly, but we don't have direct access to the data on Kafka. So it's good to have tools like Kafka Magic or Kafka Tool to access and visualize the data.
