Our primary use case for this solution is processing vast chunks of data in real-time. We deploy the solution on cloud.
Infrastructure Solutions Architect at a real estate/law firm with 10,001+ employees
Assists us in processing vast chunks of data in real-time
Pros and Cons
- "The solution can handle a high volume of data because it works and scales well."
- "It could be improved by including a feature that automatically creates a new topic and puts failed messages."
What is our primary use case?
What is most valuable?
The processing is better, and the connectors feel better than other solutions we have used. Additionally, the solution can handle a high volume of data because it works and scales well.
What needs improvement?
This solution is different from other products we have used. It could be improved by including a feature that automatically creates a new topic and puts failed messages. When we moved to this solution, we noticed that failed messages do not automatically go into Dead Letter Queue (DLQ), which we had when we used Azure. We want something similar to be included in Confluent Kafka.
The Azure Service Bus background should be improved where the message processing and messages fail and automatically go into a DLQ. In Confluent Kafka, there is no concept of DLQ, so if any messages fail, you have to handle it on your end and push it to another topic. But in Azure Service Bus, it happens automatically if any message delivery fails. It puts it in a DLQ. We didn't bother about anything on the code front, but with Confluent Kafka, there is no concept of DLQ, and you have to manage when a message fails to deliver. We have to push it to another topic manually. This should have been on the Confluent Kafka, and if anything fails, it should automatically move the message to another topic that has been created. Still, we have to do this using our code explicitly.
For how long have I used the solution?
We have been using the solution for approximately six months.
Buyer's Guide
Confluent
November 2024
Learn what your peers think about Confluent. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.
What do I think about the stability of the solution?
The solution is stable. In our limited experience, we haven't found anything that would highlight that it's not stable.
How was the initial setup?
I wasn't part of the initial setup.
What's my experience with pricing, setup cost, and licensing?
We need visibility on the licensing costs.
Which other solutions did I evaluate?
When evaluating the use case compared to the Service Bus architecture we previously used, the performance is better and more tolerant than a competitive system. Hence, because of our data volume, it can handle high-volume data, and its performance is better than other systems.
What other advice do I have?
I rate the solution nine out of ten. The solution is good, but it can be improved by including a feature that stores failed messages. I recommend this solution to new users because we went through different tech analyses and then adopted this solution. To date, we have seen improvements in performance and scalability, so we recommend this solution.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Solutions Architect at a tech services company with 10,001+ employees
Provides the data availability that we need with good features; visualization could be improved
Pros and Cons
- "Kafka Connect framework is valuable for connecting to the various source systems where code doesn't need to be written."
- "there is room for improvement in the visualization."
What is our primary use case?
We're currently working on one of our largest implementations with 500 billion-plus records in real-time and 100 billion on a daily basis. The client aggregates the reports. I'm a solution architect and we are customers of Confluent.
How has it helped my organization?
As an enterprise organization, data availability is critical and Confluent provides that SLA support. It's very important for us to have this.
What is most valuable?
KSQL is a valuable feature as is the Kafka Connect framework for connecting to the various source systems where you need not write the code. We get great support from Confluent because we're using the enterprise version and whenever there's a problem, they support us with fine-tuning and finding the root cause.
What needs improvement?
From the control center perspective, there is a lot of room for improvement in the visualization. They're now aggressively pushing the Cloud version which is good, but there are still a lot of features missing there.
For how long have I used the solution?
I've been using this solution for three years.
What do I think about the stability of the solution?
If you take the version prior to the latest one, then it will be stable. We have a team of around 10 people working on the solution. Maintenance needs an operations team and admin availability in the event of performance degradation. It needs to be monitored 24/7.
What do I think about the scalability of the solution?
This solution is highly scalable.
How are customer service and support?
Customer support is very good - we can raise a ticket through the portal and they get us on a video-sharing app to help solve the problem.
Which solution did I use previously and why did I switch?
We previously used the open-source version of Apache which doesn't have automated deployment using Ansible or even through Confluent Operator for Kubernetes. It required a lot of installation maintenance which can be tricky sometimes.
How was the initial setup?
If you go for Confluent Cloud, then the deployment is just a plug-and-play. If it's on-prem and you're looking to deploy on a non-containerized platform, then there are very good Ansible scripts, which can be completely deployed. If you're going with Kubernetes, then they have something called Confluent Operator. Deployment requires an online admin and a minimum of three people for on-prem deployment. Implementation time depends on the cluster size. For eight nodes, deployment takes around 20 minutes but for a larger cluster with around 30 nodes, it can take an hour and a half.
What other advice do I have?
I think this is a very good solution and if there's no need to specifically take the on-prem version then in terms of maintenance and scalability, the cloud version would be the preferred option. If the organization has a very high volume of data then on-prem is preferable. In terms of performance, I believe Confluent is the market leader in streaming platforms.
I rate the solution seven out of 10.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
Confluent
November 2024
Learn what your peers think about Confluent. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.
Technical Lead at a retailer with 10,001+ employees
All portfolios have access to the data that is being shared but there is a gap on the security side
Pros and Cons
- "With Confluent Cloud we no longer need to handle the infrastructure and the plumbing, which is a concern for Confluent. The other advantage is that all portfolios have access to the data that is being shared."
- "Currently, in the early stages, I see a gap on the security side. If you are using the SaaS version, we would like to get a fuller, more secure solution that can be adopted right out of the box. Confluence could do a better job sharing best practices or a reusable pattern that others have used, especially for companies that can not afford to hire professional services from Confluent."
What is our primary use case?
Our company has a wide portfolio of use cases in which we recently changed the entire landscape. We use Confluent to share useable dat across all portfolios through a central place.
We use dedicated clusters with four CKs. On standard clusters we are using just the standard configurations that comes out of the box in Confluent Cloud.
What is most valuable?
With Confluent Cloud we no longer need to handle the infrastructure and the plumbing, which is a concern for Confluent. The other advantage is that all portfolios have access to the data that is being shared.
We are still in the discovery phase. Currently, it's the introduction and acceptance phase for all the portfolios and helping them adopt the platform. Portfolios are using different features, some are using Publish-Subscribe to log events, some like the features that are similar to Kafka, while others are using ksqlDB or trying the adoption of Schema Registries.
So far we value the professional service support and the full management side of things like support with infrastructure issues.
What needs improvement?
Currently, in the early stages, I see a gap on the security side. If you are using the SaaS version, we would like to get a fuller, more secure solution that can be adopted right out of the box. Confluence could do a better job sharing best practices or a reusable pattern that others have used, especially for companies that can not afford to hire professional services from Confluent.
Another area of improvement is on the metric side. The metrics are integrated within the documentation of the metrics and implementation of that metric. This ties back to the requirement of best practices. How to use metrics in the best possible way out of the box.
If you are using Confluent on-prem or cloud-based where you are packaging it within your own cloud infrastructure, there is a lot of documentation for that making it easier. However, if you are using only the SaaS platform, there is less support from Confluent Cloud.
We are still understanding the additional features but we have seen a gap in the features of the fully managed connector. They are lesser fully managed connectors that are relevant to our business requirements so we have requested those fully managed connectors be developed and released within the Confluent Hub portfolio of connectors.
Confluence requires documentation regarding how to manage connectors that are not fully managed, including what parameters are required for each processing connector. The documentation that is provided is not easy to understand.
For how long have I used the solution?
I have been using Confluent for six months.
What do I think about the stability of the solution?
The portfolios that we have onboarded onto Confluent Cloud have been performing well. They are stable without any issues.
What do I think about the scalability of the solution?
Confluent is fairly easy to scale.
How are customer service and support?
We are using a fully managed solution. Confluent Cloud has an uptime SLA of 99.95 broad for dedicated services, dedicated clusters, and standard clusters that we are using on the cloud. It's the support that we use for the fully managed Confluent Cloud so no maintenance is required. Customer service and support have been perfect.
Which solution did I use previously and why did I switch?
We recently changed our entire landscape. In the past, every team had its own Kafka instance. Data was siloed and being used by each portfolio differently and not shared across portfolios. We wanted to find a way to share reusable data and make data sharing by default so that it was used across all portfolios and could be leveraged through a central place. We wanted to move away from a self-managed Kafka system and make it fully managed. That's why we started looking at Confluent Cloud.
How was the initial setup?
We are not using the Confluent Platform. How we deploy is by onboarding people to connect to the Confluent Cloud, it's just a networking connection into the Confluent Cloud cluster, not a deployment pipeline.
What's my experience with pricing, setup cost, and licensing?
From a costing perspective, there are costing implications. If you go the cloud route, as you scale there will be cost implications on a per hour basis. Confluent Cloud is more expensive, be mindful of that.
What other advice do I have?
The main point to understand is that Confluent Platform and Confluent Cloud are two different offerings. Confluent Platform requires you to self-manage or manage the Confluent Platform within your cloud environment, rather than Confluent Cloud which uses your applications that are deployed in your cloud to connect to Confluent Cloud.
When you decided to get into Confluent, have clarity on which route you want to go at the outset. Secondly, if you choose Confluent Cloud which is the SaaS platform, have your implementation plan mapped out before you go ahead. Lastly, have your naming conventions, implementation, and securing your topics, your service account or ACLs are all mapped out well.
Support is a 10 out of 10, but overall I would rate Confluent a 7 out of 10.
Which deployment model are you using for this solution?
Hybrid Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
General Counsel of Corporate at Kakao Corp
Is scalable and has storage clearance features
Pros and Cons
- "I would rate the scalability of the solution at eight out of ten. We have 20 people who use Confluent in our organization now, and we hope to increase usage in the future."
- "The Schema Registry service could be improved. I would like a bigger knowledge base of other use cases and more technical forums. It would be good to have more flexible monitoring features added to the next release as well."
What is our primary use case?
We collect service log data with Confluent and use it for fraud detection in game services.
How has it helped my organization?
We are a game service company, and Confluent has helped us improve our quality of service.
What needs improvement?
The Schema Registry service could be improved.
I would like a bigger knowledge base of other use cases and more technical forums. It would be good to have more flexible monitoring features added to the next release as well.
For how long have I used the solution?
I've been using Confluent for about two years.
What do I think about the stability of the solution?
For stability, I would give this solution a rating of seven out of ten.
What do I think about the scalability of the solution?
I would rate the scalability of the solution at eight out of ten. We have 20 people who use Confluent in our organization now, and we hope to increase usage in the future.
How are customer service and support?
I would rate the solution's technical support at seven out of ten.
How would you rate customer service and support?
Neutral
How was the initial setup?
On a scale from to ten, where one is difficult to set up and ten is easy to set up, I would rate this solution's initial setup at six. It was quite difficult.
The deployment took about a week. We had a team of four people for the deployment. A team of two handle the maintenance.
What's my experience with pricing, setup cost, and licensing?
On a scale from one to ten, where one is low pricing and ten is high pricing, I would rate Confluent's pricing at five. I have not encountered any additional costs.
Which other solutions did I evaluate?
We evaluated Kinesis and MSK, but chose Confluent because we wanted to use its RBAC feature. Also, the other solutions did not have a feature to clear storage.
What other advice do I have?
I would recommend Confluent as an alternative to those who are unfamiliar with Kafka. Overall, I would rate Confluent at eight out of ten.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Senior Distributed Platform Engineer at a tech services company with 201-500 employees
Beneficial data replication, effective self balancing, and stable
Pros and Cons
- "The most valuable feature that we are using is the data replication between the data centers allowing us to configure a disaster recovery or software. However, is it's not mandatory to use and because most of the features that we use are from Apache Kafka, such as end-to-end encryption. Internally, we can develop our own kind of product or service from Apache Kafka."
- "They should remove Zookeeper because of security issues."
What is our primary use case?
I work in the insurance domain. We stream data in and out Confluent. If financial data is being used most organizations prefer to stay on-premise with their data, but there is a cloud version available. These are the features that Confluent provides.
How has it helped my organization?
Confluent has benefited our company because it provides granular access, enterprise support, self-balancing cluster. If there is any data skewness or one server is busy doing some processing, it shifts to the other processor and balances the data between them. There are auto data balances, self-balancing clusters, replication, and role-based access controls.
What is most valuable?
The most valuable feature that we are using is the data replication between the data centers allowing us to configure a disaster recovery or software. However, is it's not mandatory to use and because most of the features that we use are from Apache Kafka, such as end-to-end encryption. Internally, we can develop our own kind of product or service from Apache Kafka.
What needs improvement?
They should remove Zookeeper because of security issues.
For how long have I used the solution?
I have used Confluent within the last 12 months.
What do I think about the stability of the solution?
The latest version of 6.2.0, has almost everything and it's stable.
What do I think about the scalability of the solution?
The solution is scalable. If you have good hardware, especially for Kafka then it will perform very well. It's easy to scale up, but there are issues scaling down, which may result in losing data.
I don't know exactly how business is doing. We currently are supporting, and we are running up to the threshold of the solution. I'm not sure about the future if we will increase usage.
We have approximately 250 applications running.
How are customer service and support?
We have contacted technical support. We create support tickets, in case of emergencies or when any support is required.
I've been working with the support team for four years. They are good. The level one team will go around the issue by gathering information, but the next level, and the third level development team, are very good.
How was the initial setup?
The installation is straightforward and we use our own ansible-playbook, which was designed similar to CP ansible which is supported by Confluent.
What about the implementation team?
We have two people that do the implementation. We manage four clusters which include integration tests, reproduction, and production. We did the installation with some simple scripts and designing the script takes a couple of weeks.
Confluent requires maintenance because we might see issues with the applications and need to support them. We need to keep checking the performance and we should keep on monitoring them because high critical data will be passing through. There is a need for regular OS patching and any upgrades that Confluent provides.
Not everybody will be aware of Kafka's technology which means we educate the developers. For example, the properties to be used and what makes optimum performance from the solution.
What was our ROI?
I have had a return on investment by using Confluent. It's faster than any other technology in real-time. You don't have to wait for your transactions to happen overnight.
What's my experience with pricing, setup cost, and licensing?
The license Confluent is server-based. It depends on the number of services that you are running in the cluster.
Confluent is expensive, I would prefer, Apache Kafka over Confluent because of the high cost of maintenance.
Large organizations, prefer to use open-sourced solutions because they provide a cost reduction. Apache Kafka is open source and there are many other open-source security tools, which can be used that provide features that Confluent offers.
Which other solutions did I evaluate?
I have evaluated Apache Kafka.
What other advice do I have?
Confluent has improved a lot and they're currently working on version 7.0 where they're removing Zookeeper, which is highly vulnerable to security attacks.
My advice to others wanting to use this solution is for them to use automation wherever it is required and it is a very stable product, not many issues arise If you know how to manage them. Overall, it's a great company and they have excellent software.
I rate Confluent a nine out of ten.
Which deployment model are you using for this solution?
On-premises
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
Provides good features like message queues and event-driven architecture
Pros and Cons
- "We mostly use the solution's message queues and event-driven architecture."
- "Confluent has a good monitoring tool, but it's not customizable."
What is our primary use case?
We are providing the solution for real-time data for the banking sector.
What is most valuable?
We mostly use the solution's message queues and event-driven architecture. Since Confluent has integrated with Kafka, we can utilize its fault tolerance and matrices.
What needs improvement?
Confluent is expensive, and we face certain limitations while using the tool. Confluent has a good monitoring tool, but it's not customizable. We can only use the existing dashboards. On the other hand, we can utilize maximum customization while using Grafana with Prometheus. Confluent also supports JMX metrics, but using Grafana with Prometheus is very effective.
For how long have I used the solution?
I have been using Confluent for two months.
What do I think about the stability of the solution?
I rate the solution’s stability a nine out of ten.
What do I think about the scalability of the solution?
Confluent is a very scalable solution. Since we provide solutions for the banking sector, we need a high throughput of massive data per day.
I rate the solution a seven out of ten for scalability.
How are customer service and support?
Confluent provides really good technical support for its partners.
How would you rate customer service and support?
Positive
Which solution did I use previously and why did I switch?
I have previously worked with other solutions like MuleSoft Anypoint, Dell Boomi, and Redpanda.
What about the implementation team?
We are currently doing some POCs, which take two to three days based on availability. Sometimes, if the technical people are unavailable, I have to wait for their response and get into meetings.
What's my experience with pricing, setup cost, and licensing?
Confluent is an expensive solution.
On a scale from one to ten, where one is cheap and ten is expensive, I rate the solution's pricing an eight out of ten.
What other advice do I have?
Confluent can be easily integrated with other systems. Most clients don't know the actual use cases of Confluent or Kafka. We explain the real use cases to the clients, like fault tolerance, high throughput, low latencies, and monitoring tools.
I joined my company just one month ago. Working with Confluent for the first time was a bit hard for me because it has a lot of documentation for everything. I need some technical experts to utilize the solution's functionalities. Maintaining the solution is easy because we are part of Confluent, and they are available.
Overall, I rate the solution an eight out of ten.
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
Last updated: Apr 5, 2024
Flag as inappropriateBusiness Analyst at Infor
A great tool for business analysts and project managers to create fast documentation
Pros and Cons
- "The documentation process is fast with the tool."
- "The product should integrate tools for incorporating diagrams like Lucidchart. It also needs to improve its formatting features. We also faced issues while granting permissions."
What is our primary use case?
I use the tool to create documentation that needs particular specifications.
What is most valuable?
The documentation process is fast with the tool.
What needs improvement?
The product should integrate tools for incorporating diagrams like Lucidchart. It also needs to improve its formatting features. We also faced issues while granting permissions.
For how long have I used the solution?
I have been working with the tool for three years.
What do I think about the stability of the solution?
The tool is stable.
What do I think about the scalability of the solution?
The solution is scalable. My company has more than 25,000 users of the tool.
How was the initial setup?
The tool’s initial setup is complicated.
What other advice do I have?
I would rate the overall solution a nine out of ten. Confluent is a great tool for business analysts and project managers.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Tech manager at a tech services company with 1,001-5,000 employees
Helps to stream event data from APIs and work with CDC log data from databases
Pros and Cons
- "Our main goal is to validate whether we can build a scalable and cost-efficient way to replicate data from these various sources."
- "One area we've identified that could be improved is the governance and access control to the Kafka topics. We've found some limitations, like a threshold of 10,000 rules per cluster, that make it challenging to manage access at scale if we have many different data sources."
What is our primary use case?
We use Apache Kafka and Confluent Cloud to stream event data from APIs and work with CDC log data from databases for data integration.
What is most valuable?
Our main goal is to validate whether we can build a scalable and cost-efficient way to replicate data from these various sources.
What needs improvement?
One area we've identified that could be improved is the governance and access control to the Kafka topics. We've found some limitations, like a threshold of 10,000 rules per cluster, that make it challenging to manage access at scale if we have many different data sources.
For how long have I used the solution?
I have been using the product for three months.
What do I think about the stability of the solution?
I haven't encountered any major stability or performance problems so far.
What do I think about the scalability of the solution?
The platform seems very scalable in handling high volumes of data and throughput.
How are customer service and support?
The tool's support could have provided more guidance and best practices, especially when designing our solution. Instead, we discovered and solved many of the challenges ourselves. The support has been responsive, but I think it could be better.
How would you rate customer service and support?
Neutral
Which solution did I use previously and why did I switch?
I've also worked a bit with AWS MSK, a competing product for Confluent Cloud. From a maintenance standpoint, I think Confluent requires less effort than something like MSK or running Kafka on Kubernetes ourselves.
How was the initial setup?
The general setup and deployment of Confluent are very easy.
What's my experience with pricing, setup cost, and licensing?
Regarding pricing, I think Confluent is a premium product, but it's hard for me to say definitively if it's overly expensive. We're still trying to understand if the features and reduced maintenance complexity justify the cost, especially as we scale our platform use.
What other advice do I have?
I rate the overall solution an eight out of ten.
Which deployment model are you using for this solution?
Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Last updated: Aug 21, 2024
Flag as inappropriateBuyer's Guide
Download our free Confluent Report and get advice and tips from experienced pros
sharing their opinions.
Updated: November 2024
Product Categories
Streaming AnalyticsPopular Comparisons
Databricks
Azure Stream Analytics
Amazon Kinesis
Apache Flink
Amazon MSK
Google Cloud Dataflow
Spring Cloud Data Flow
PubSub+ Platform
Apache Spark Streaming
Starburst Enterprise
Cloudera DataFlow
Apache Pulsar
Aiven Platform
Starburst Galaxy
IBM Streams
Buyer's Guide
Download our free Confluent Report and get advice and tips from experienced pros
sharing their opinions.
Quick Links
Learn More: Questions: