Try our new research platform with insights from 80,000+ expert users
reviewer1218324 - PeerSpot reviewer
Head of Technology - Money Movement Platform at a financial services firm with 10,001+ employees
Real User
Feature rich, highly scalable, and straightforward to implement
Pros and Cons
  • "All the features of Apache Kafka are valuable, I cannot single out one feature."
  • "Prioritization of messages in Apache Kafka could improve."

What is our primary use case?

We use Apache Kafka primarily to queue the transactions or total the transactions.

How has it helped my organization?

Apache Kafka has helped our organization handle larger volumes without affecting the infrastructure load.

What is most valuable?

All the features of Apache Kafka are valuable, I cannot single out one feature.

What needs improvement?

Prioritization of messages in Apache Kafka could improve.

Buyer's Guide
Apache Kafka
November 2024
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
817,354 professionals have used our research since 2012.

For how long have I used the solution?

I have been using Apache Kafka for approximately six years.

What do I think about the stability of the solution?

The stability of Apache Kafka is very good.

What do I think about the scalability of the solution?

Apache Kafka is the most scalable solution in the market.

How are customer service and support?

I have not used the support from Apache Kafka.

How was the initial setup?

Apache Kafka is straightforward to implement.

What about the implementation team?

We did the implementation of Apache Kafka in-house.

Which other solutions did I evaluate?

I did not evaluate other solutions.

What other advice do I have?

I rate Apache Kafka a nine out of ten.

Which deployment model are you using for this solution?

Hybrid Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Technical Director at Metrofibre Networx
Real User
Top 20
A reliable and stable stream-processing platform with a good customer support team
Pros and Cons
  • "As a software developer, I have found Apache Kafka's support to be the most valuable...The solution is easy to integrate with any of our systems."
  • "The solution should be easier to manage. It needs to improve its visualization feature in the next release."

What is our primary use case?

We have a camera monitoring security system, in which we post messages onto the queue, which involves various steps in processing the message, like checking for the number of clients, running it against the police data, etc. So Apache Kafka is a security application with many types of consumers. We set up a workflow system with different sites, which works well.

What is most valuable?

As a software developer, I have found Apache Kafka's support to be the most valuable. The support team sends available information regarding the library and how to use the plugins. The solution is easy to integrate with any of our systems. We have other alternatives, but this is the one that seems to be the most popular database support.

What needs improvement?

The solution should be easier to manage. It needs to improve its visualization feature in the next release.

For how long have I used the solution?

I have been using it for three years.

What do I think about the stability of the solution?

It is a stable solution. We never faced any issues. I rate it a ten out of ten.

What do I think about the scalability of the solution?

It is a scalable solution. We set up a category with different consumers balancing things, which works as I thought.

How are customer service and support?

I did not contact the technical support as it was not required.

Which solution did I use previously and why did I switch?

We used Linksys for visualization along with Confluence, but there needed to be more value. For us, Apache Kafka is the best solution based on the support and third-party systems as it builds our subsystems around because we have a lot of development teams.

How was the initial setup?

The initial setup was straightforward because I've got a lot of experience in this field. But even for a junior person, it would be fine. There are so many resources, and it's very well documented as they are a premium service provider. So it makes the setup just easier.

The deployment takes a few days.

We set up a free cluster for this service because we use a lot of data. We use ZooKeeper to secure different products for instruction with the cluster. But, it was easy as it is a popular product, and much information is available. It can download data, like fifty gigs per day. We can effectively handle it all as well. I never developed any issues.

What's my experience with pricing, setup cost, and licensing?

It's a premium product, so it is not price-effective for us.

What other advice do I have?

Apache Kafka is an out-of-the-box, reliable solution. For people in the fiber business, we need a reliable solution, and this solution is hundred percent reliable. If it is set up correctly, it hardly has any issues due to the more extensive user base; even if there are issues, it is sorted by the community. I rate it nine out of ten.

Which deployment model are you using for this solution?

Private Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Apache Kafka
November 2024
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
817,354 professionals have used our research since 2012.
Felipe Lopes - PeerSpot reviewer
Engineering Manager at Alice
Real User
You can receive and distribute data in real-time
Pros and Cons
  • "I have seen a return on investment with this solution."
  • "I suggest using cloud services because the solution is expensive if you are using it on-premises."

What is our primary use case?

The primary use case of the solution is for asset communication through our microservices.

How has it helped my organization?

The solution has allowed us to take the use cases provided by another communication tool and resolve those issues.

What is most valuable?

The most valuable feature is how persistent it is. For example, we are able to reprocess messages when we need to, we're able to recover methods to consume them.

What needs improvement?

The solution can be improved by reducing the cost to run it on the premises.

For how long have I used the solution?

I have been using the solution for four years.

What do I think about the stability of the solution?

The stability of the solution is good.

What do I think about the scalability of the solution?

The solution is scalable.

How was the initial setup?

The initial setup was straightforward.

What about the implementation team?

The implementation was through a vendor.

What was our ROI?

I have seen a return on investment with this solution.

What other advice do I have?

I give the solution a nine out of ten.

We have 80 people using the solution and five people are required to maintain it.

I suggest using cloud services because the solution is expensive if you are using it on-premises.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
reviewer1421481 - PeerSpot reviewer
Solution Architect at a manufacturing company with 10,001+ employees
Real User
Good performance when a high throughput is required, but they need to implement a portal
Pros and Cons
  • "The processing power of Apache Kafka is good when you have requirements for high throughput and a large number of consumers."
  • "They need to have a proper portal to do everything because, at this moment, Kafka is lagging in this regard."

What is our primary use case?

I am a solution architect and I used Apache Kafka in this role.

What is most valuable?

The processing power of Apache Kafka is good when you have requirements for high throughput and a large number of consumers. 

What needs improvement?

They need to have a proper portal to do everything because, at this moment, Kafka is lagging in this regard. It could be used to do the preprocessing or the configurations, instead of directly doing it on the queues or the topics. If you look at Solace, for example, they have come up with a portal where you don't need to touch these activities. You don't need to access the platform beyond the portal.

For how long have I used the solution?

I have used Apache Kafka for between one and one and a half years.

What do I think about the stability of the solution?

Apache Kafka is stable.

What do I think about the scalability of the solution?

This is certainly a scalable product. There are currently 30 or more people using it but we expect to scale beyond this. It is going to be an enterprise tool within the company.

How are customer service and technical support?

I am not directly interacting with the service people at this moment. It is limited for now because we are still exploring and effecting our architecture and design, and deciding how to align it with our existing strategy. There is not much progress in this regard and it will take more time.

Which solution did I use previously and why did I switch?

Prior to working with Apache Kafka, there was no messaging queue system. For many projects, they were using the Azure Event Hub, but it was not serving the purpose. So, we started moving towards Kafka, and that's why we have procured Confluent Kafka.

Several months ago, I stopped working on Apache Kafka. I am now working on Confluent Kafka. It was not my decision to switch solutions.

My current organization has chosen Confluent Kafka for various reasons. One is that we have a large number of streaming requirements, and Confluent Kafka has one more layer on top of Apache Kafka to do this transformation and connecting with other multiple lane systems.

There are out-of-the-box features along with the KSQL features. For example, things like fetching the events are kind of query-based. So, that seems to be a good feature for our requirements. That is why we ultimately procured Confluent Kafka.

For some time, I have also worked with Solace and it has an advantage. Given that my core strength is integration, I work with integration platforms such as MuleSoft, Azure functions, then TIBCO. Based on our requirements, I found that the event-driven APA implementation with Solace was easier.

Solace also has a top-notch solution for portal management and you register your producers, consumers, and preprocessing logic. All of these things are pretty easy to do. This is an area where Kafka could use some enhancement.

How was the initial setup?

I don't think that the initial setup was a complex process.

Which other solutions did I evaluate?

MQ messaging systems are not my core strength but for any integration platform where we have a large number of APIs and events, to integrate with an IoT platform, for example, I found Kafka is better than ActiveMQ.

I'm not getting into in MQTT or other things but comparatively, when you compare ActiveMQ and Kafka, Kafka has done better.

What other advice do I have?

I think that many people are using Apache Kafka just as a publishing and subscription model, but I feel that Kafka is better than that. Furthermore, Confluent Kafka is even more than that.

Confluent Kafka is offering features that are equal to those of a data lake. You can do lots with data, and huge data can be persisted. However, many people are not using that feature. Rather than make use of persistence logic, they are pushing the messages and consuming them. Maybe if people were using it for persistence, they would see the impact or real power of Kafka.

I would rate this solution a seven out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Roger Sabourin - PeerSpot reviewer
Roger SabourinSenior Manager, Analyst Relations at a tech vendor with 201-500 employees
Real User

You're in luck, Solace's PubSub+ Event Portal for Kafka does all the things you're looking for, specifically for your Kafka environments, be they open source Kafka, Confluent or Amazon MSK.  Check it out, or request a free trial at https://solace.com/products/po...

reviewer1247268 - PeerSpot reviewer
Technology Lead at a tech services company with 10,001+ employees
MSP
Top 20
A cost-effective solution for high volume, multi-source data collection
Pros and Cons
  • "The most valuable feature is that it can handle high volume."
  • "Kafka does not provide control over the message queue, so we do not know whether we are experiencing lost or duplicate messages."

What is our primary use case?

Our company provides services and we use Apache Kafka as part of the solution that we provide to clients.

One of the use cases is to collect all of the data from multiple endpoints and provide it to the users. Our application integrates with Kafka as a consumer using the API, and then sends information to the users who connect. 

What is most valuable?

The most valuable feature is that it can handle high volume.

Apache Kafa is open-source and some of our clients are interested in becoming more involved in that.

What needs improvement?

Kafka does not provide control over the message queue, so we do not know whether we are experiencing lost or duplicate messages. Better control over the message queue would be an improvement. Solutions such as ActiveMQ do afford better control. Because of this, there is sometimes a gap in the results where we have either lost messages, or there are duplicates.

We have had problems when there was an imbalance because all of the messages were being sent back.

For how long have I used the solution?

I'm a beginner with Apache Kafka.

What do I think about the stability of the solution?

I cannot judge stability without having better control over the message queue, although I feel that it is not 100% stable. 

How are customer service and technical support?

We have not been in contact with technical support. For our first implementation with it, Kafka was already set up and running. When we did our PoC, I was not part of the team who was facing issues and it was they who were in contact with support.

Which solution did I use previously and why did I switch?

I also have experience with IBM MQ.

How was the initial setup?

We had problems when we were setting up Kafka ourselves to conduct our PoC internally. Kafka would not start and it was related to parameters or property settings in Java. We were able to work around it, but we had problems like adding certificates.

What about the implementation team?

In one case, we were using Kafka after it had already been set up, externally. It worked fine and we just had to configure some of the connectors that we wanted to try out.

What's my experience with pricing, setup cost, and licensing?

Apache Kafka is open-source and can be used free of charge.

What other advice do I have?

In this type of solution, you need to be able to accept a high volume of messages, but not lose any, and not have any duplicates. Because we are unable to control the queue in Kafka, I cannot say that this works 100%.

The suitability of this solution depends on the use cases. There are two or three things that we are worried about, and we will be very careful in choosing solutions. In cases where the messages are well organized, or there is no worry that there will be duplicate or dropped messages, then I recommend using Kafka. Also, I recommend this solution for those looking to get involved with open-source applications.

Other than the problems with having no control over the queue, Apache Kafka is wonderful.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Paul Adams - PeerSpot reviewer
Consultant Solution Architect at a tech services company with 51-200 employees
Consultant
Top 20
Straightforward implementation, highly resilient, and good support
Pros and Cons
  • "The most valuable feature of Apache Kafka is its versatility. It can solve many use cases or can be a part of many use cases. Its fundamental value of it is in the real-time processing capability."
  • "Managing Apache Kafka can be a challenge, but there are solutions. I used the newest release, as it seems they have removed Zookeeper, which should make it easier. Confluent provides a fully managed Kafka platform, in which the cluster does not need to be managed."

What is our primary use case?

We had an application stack consisting of Salesforce frontend and a Commander VPN position management system and used Apache Kafka to decouple the microservices. Additionally, we planned to use Kafka for stream processing and to use event sourcing to pull data from legacy systems and reference data to form a compacted topic that the microservices could consume.

The usage of Kafka is a combination of deploying on a personal Kubernetes cluster or using a managed service such as MSK. However, most people who use Kafka are using a managed service provided by Confluent. It can be deployed on the cloud or on-premise.

What is most valuable?

The most valuable feature of Apache Kafka is its versatility. It can solve many use cases or can be a part of many use cases. Its fundamental value of it is in the real-time processing capability.

You need time-sensitive technology now, particularly in the analytics space. We have looked at using change data capture and Apache Kafka to modernize our analytics capabilities. Additionally, microservices can be used to capture events from legacy systems.

What needs improvement?

Managing Apache Kafka can be a challenge, but there are solutions. I used the newest release, as it seems they have removed Zookeeper, which should make it easier. Confluent provides a fully managed Kafka platform, in which the cluster does not need to be managed.

If it is a native Apache Kafka, it would have schema registry capabilities. However, this type of functionality is often provided by third-party tools. Additionally, there may be a need for improved manageability and additional tools to manage the cluster, including standard operational metrics and inbuilt management capabilities.

For how long have I used the solution?

I have been using Apache Kafka for approximately three years.

What do I think about the stability of the solution?

The solution is highly resilient.

I rate the stability of Apache Kafka a nine out of ten.

What do I think about the scalability of the solution?

Apache Kafka is scalable.

I rate the scalability of Apache Kafka a nine out of ten.

How are customer service and support?

The support from Apache Kafka is good.

How was the initial setup?

The initial setup of Apache Kafka is easy to set up a cluster.  I did the initial setup on my laptop and it is straightforward. I used the Confluent version, but even if you want to run native capabilities it's straightforward to do the implementation.

What about the implementation team?

The recent proof of concept was done on behalf of a client by a system integrator. Similarly, the previous one was mainly done in-house and it utilized Confluent, Apache Kafka, and MSK. The process involved setting up pre-built capabilities.

What's my experience with pricing, setup cost, and licensing?

The price of the solution is low.

I rate the price of Apache Kafka a nine out of ten.

What other advice do I have?

I rate Apache Kafka a nine out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: My company has a business relationship with this vendor other than being a customer: Integrator
PeerSpot user
Director at Tibco
Real User
The solution is stable, scalable, and open-source
Pros and Cons
  • "The open-source version is relatively straightforward to set up and only takes a few minutes."
  • "The solution can improve its cloud support."

What is our primary use case?

We have got this product, which is meant for integration. So our use cases are essentially integrating with other systems, using any messaging stack. We use these products in Dev and QA and we have connectors for various different messaging applications. Apache Kafka just happens to be one of the messaging applications that we connect with. We also have our own messaging, it's called Enterprise Messaging Server and Rendezvous, we connect to those also. Our product is essentially used for integration. So we connect to almost all messaging applications.

What is most valuable?

The most valuable feature is the speed at which the solution can be deployed.

What needs improvement?

The solution can improve its cloud support.

For how long have I used the solution?

I have been using the solution in Dev and QA for a few years.

What do I think about the stability of the solution?

The solution is stable.

What do I think about the scalability of the solution?

The solution is scalable.

Which solution did I use previously and why did I switch?

Since we are supporting various different messaging applications, we tend to use and support all the messaging applications that are popular. Like SQS, Google pops up, Active MQ, Rapid MQ, MQTT, and IBM MQ.

How was the initial setup?

The open-source version is relatively straightforward to set up and only takes a few minutes.

What about the implementation team?

We typically implement the solution in-house.

What's my experience with pricing, setup cost, and licensing?

The solution is open source.

What other advice do I have?

I give the solution an eight out of ten.

We test all the supported versions of the solution based on our customers' use.

We support our integration product. So we need to do dev and QA with Apache Kafka or any other messaging applications. But we do not provide support. The solution can be supported by someone else.

We don't need to have any specific staff for deployment. All the developers in QA can install and configure the solution. We don't have a separate person for maintenance.

Our team and our product dev and QAs all use the solution.

I think Apache Kafka is a good solution and I recommend it to others.

Which deployment model are you using for this solution?

Hybrid Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Other
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Assistant Professor at CHAROTAR UNIVERSITY OF SCIENCE AND TECHNOLOGY
Real User
Difficult to configure, lacking automation, but has good community support
Pros and Cons
  • "The valuable features are the group community and support."
  • "The solution can improve by having automation for developers. We have done many manual calculations and it has been difficult but if it was automated it would be much better."

What is our primary use case?

We are in the early stages of testing this solution in our lab as a demo. It is in development and we are not in production at this point.

We are using this solution to relay events when they happen to multiple receivers at once to allow better functionality.

How has it helped my organization?

Apache Kafka has helped our client's online restaurant company by allowing them to take any orders and send the notifications with some other details, such as logic commands, to the different microservices.

What is most valuable?

The valuable features are the group community and support.

What needs improvement?

The solution can improve by having automation for developers. We have done many manual calculations and it has been difficult but if it was automated it would be much better.

For how long have I used the solution?

I have been using this solution for approximately three months.

What do I think about the scalability of the solution?

The solution's scalability is important for our ability to have more throughput from multiple receivers. If we need more throughput it can deliver.

Which solution did I use previously and why did I switch?

We did use other solutions previously but this solution makes things a lot easier.

How was the initial setup?

The installation is fairly easy. Additionally, there is a cloud-based version available if a use case requires it.

What about the implementation team?

We did the implementation ourselves.

What's my experience with pricing, setup cost, and licensing?

The solution is free, it is open-source.

What other advice do I have?

There is a lot of configuration involved in this solution. We have found many configurations that have helped us but it would be beneficial if there was automation. 

I rate Apache Kafka a five out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free Apache Kafka Report and get advice and tips from experienced pros sharing their opinions.
Updated: November 2024
Product Categories
Streaming Analytics
Buyer's Guide
Download our free Apache Kafka Report and get advice and tips from experienced pros sharing their opinions.