Apache Kafka Reviews

We deployed this solution in a project for one of our customers to synchronize the different applications; to transport information from one application to another. I'm a program manager and we are customers of Apache.

This solution is robust and delivers messages quickly. It's a simple but good product.

The management tool could be improved.

I've been using this solution for the past few months. 

In this project, we used Kafka to synchronize 28 nodes spread out nationally and it seems scalable. We plan to consolidate the 28 nodes for national integration of nodes and schemas.

We didn't need to contact technical support. We just allotted the software, installed it and started working with it. We carried out a lot of testing pre-development. Development was done with a company that previously used Kafka so we were able to exchange technical information. 

We previously used ActiveMQ for another project.

The initial setup was a little complex. We carried out the development ourselves. 

Our clients purchased the license and they think it's an affordable solution. 

I recommend this solution, we're probably going to use it again in another project.

I rate this solution eight out of 10. 

I am a user, as well as an integrator for our clients. This is one of the products that we implement for others.

The most valuable features of this solution is the architectural style of messaging or event streaming. First is important to understand that anything can be represented as an event=message e.g new order, status change, confirmation, information from IoT or from monitoring system. Each message can be transport very quickly from the source (producer) into consumer(s). Messages can be changed in the fly - streaming messaging.

Messages can be process exactly at once or at least once. Ordering of messages depends on just configuration setup. 

When comparing it with other messaging and integration platforms, this is one of the best rated. Message store is out of the box functionality. Messages are automatically stored based on parameter setup of retention policy. Messages can remain for longer, which can be configured from a few milliseconds up to years. Scalability and availability of messages can be changed with zero maintenance window.

You don't need extra clusters to achieve high availability for the messaging system like Veritas, PowerHA or other.

One platform for classical messaging, real time messaging, ETL, message streaming. 

ETL can be realised with connectors into external systems which also could run in more instances. Exists a lot of ready to use connectors and new ones can be developed.

The model where you create the integration or the integration scenario needs improvement. It contains fewer developer words or maintaining words where someone prepared the topics, the connectors, or the streaming platforms. You would first need to have a control center from a third party for managing. 

If you would like to prepare something that is a more sophisticated integration scenario, where you use one microservice to provide the event or a second to several that consumed these microservices, then this needs to be modeled elsewhere. 

Also, when comparing to the traditional ESD for data mixing, you can create a scenario that could be deployed with inputs and some outputs.

Most business like the topics, but for me, I think that it is a problem that messaging platforms have, there is no design tool with IDE for creating.

It would be helpful to create a more complex solution for several types of styles, and not just for one provider or for one customer. That would be easier, but if you have more than one consumer then it could be a more complex scenario. It would be like events that go to several microservers to create orders, validate orders, and creating words. This would be helpful.

In the next release, adding some IDE or developing tools, for creating better integration scenarios, even though it already a developer-oriented solution, would be helpful. It would also be helpful for the auto-deployment.

Having a governance style would also be helpful to understand. 

It would be beneficial to have a repository of all of the topics, data types that exist, or data structures.

I have been working with this solution for one year.

It's a stable solution.

Kafka is very scalable, which is an important feature of it.

Our clients have approximately ten applications in their companies that communicate with Kafka.

I have not contacted technical support through Kafka, I communicate directly with Confluence. Confluence is the company that developed the open-source platform and they provide support.

The communication is very good and they are very capable of assisting you with all technical inquiries.

There are direct contacts that make it easy.

Previously, I worked with IBM MQ, a different type of messaging platform.

For the most part, the initial setup is easy, but if you need a more sophisticated infrastructure or if you have to set up the topics, then you have to be careful and you have to be more knowledgable in Kafka. You will have to know the parameters for the rotations, the size of the message, and the timeouts, as an example.

For a developer it is easy, but for an administrator and production, it requires more.

Apache Kafka is an open-source solution and there are no fees, but there are fees associated with confluence, which are based on subscription.

I would recommend trying this solution. 

Take the time to understand it because it is a different style when it comes to working with data.

I would rate this solution a nine out of ten.

• Message Retention: Unlike regular message queues, messages stay in Kafka after clients consume them. A message can be consumed over and over again by the same or a different client until topic retention (by max data size or oldest message timestamp) kicks in and the oldest messages get deleted. This can be very handy in many scenarios: handling bugs in software, testing code, simple distribution of message processing, and routing messages to many different consumers simultaneously.
• Horizontal Scalability: To add more capacity, both in terms of storage and performance to a Kafka cluster, you just need to add more servers. Regular message queues usually work in a master-slave configuration and do not scale very well horizontally.
• Simplicity in operations.

It has become dead simple to connect different application and services, saving a lot of development hours.

The standard Kafka Java library, which is shipped with the product, is too complex for inexperienced users. At my company, engineering teams ended up writing wrapper libraries to solve complex issues. Kafka client libraries in general are complex, regardless of language. This is the price Kafka users have to pay for having simple, yet robust, server-side code.

What could be improved is the hard dependency on ZooKeeper. The work in this direction has already been started, though. Overall, the project is moving forward at a very good pace

I have used Kafka for three years.

Sometimes we have stability issues, but not often.

We have not had any scalability issues.

There is no official technical support as the product is 100% open source.

We used RabbitMQ before. It does not scale well.

The setup was pretty straightforward.

There is no pricing and licensing.

We didn't evaluate any other options.

Go ahead. It's a great product.

Real-time streaming and persistence into distributed nodes. It provides a simple mechanism to create, publish, and subscribe.

We are using Kafka as part of our product. It is one of the messaging layers used to interact between various layers of software modules. This provides a clear separation of modules and leverages it for development and testing of different modules.

The management tools are getting mature. When we have thousands of topics, it is hard to visualize.

I’ve been using Kafka for two years.

We have not encountered any stability issues.

We have to balance the nodes when topics partition across cluster nodes. As it assumes they are of equal sizes, sometimes some nodes may not be allocated similar resources. Reassignment moves all the partitions of specified topics which may be an issue when not planned for.

We have the source code to make changes if necessary.

Kafka rendered itself suitable for our product offering. It supports all the necessary requirements for a real-time pipeline.

Setting up was easy with ZooKeeper.

With paid support from Confluent, you get the additional benefit of Kafka Connect.

We used Akka Streams for faster communication, but it would require additional configuration and setup for persistence. Kafka provides those by default.

Kafka provides distributed persistence and streaming layers. The user has flexibility in managing as a consumer on how to consume messages if they have to handle resilience in their code. It requires ZooKeeper.

One of the best features which I have worked with is replay.

Real-time log aggregation which was earlier done with rsync has been moved to Kafka infrastructure along with other real-time streams.

• GUI for Kafka infrastructure monitoring and deployment

I have used it for two years.

Documentation is quite comprehensive.

I found it very stable.

No issues with scalability.

We used the open-source version.

We used the open-source version.

We previously used rsync, which was not real-time.

Initial setup was mostly intuitive (based on rsync).

Implementation was in-house based on the open-source version.

Target was to achieve real-time service.

Before choosing this product, we did not evaluate other options.

The most valuable features are topic-based eventing, scalability, and retention periods.

My organization is transforming by using the new SOA/eventing-based architecture. The application depends on the employees’ information events. Kafka is very helpful in implementing this. It increases the performance and gives the details to multiple external/internal teams using Kafka topics in an asynchronous manner.

For example, if someone is moving from one office to another one, we have to update the software. While updating it, the system puts that event in a topic so that all other consumers can update that person’s new location. This can include the payroll team, the insurance team, and the hospital network.

The retention period helps us retain the data in the topic for the configured number of days. In this example, if any of the consumers fail to consume the message from the topic, then that message will be there until the retention period ends.

I would like to see a more user-friendly GUI.

We have used this solution since December, 2015.

If you are using the same group ID for multiple topics, it may shut down the application. We have faced this issue before.

We have not had any scalability issues.

I would give technical support a rating of 6 out of 10.

We were using ActiveMQ, which is just a messaging system. We are changing because of Kafka’s added value of scalability, retention, and high payload support.

The installation was somewhat straightforward.

The solution is worth the money.

This is the best tool I have ever used for asynchronous, event-based solutions.

We are a service implementer and we supply this solution to our customers. I'm a company co-founder and we are customers of Apache. 

The solution has improved our functionality, it's one of the best streaming platforms I've used. 

I'd like to see improvement in terms of observability.

I've been using this product for the last five years on and off. 

The solution is stable. 

The solution is scalable.

The initial setup is straightforward, it's not complicated.

This is an open-source product.

I rate this solution nine out of 10. 

Apache Kafka is used for connecting components between each other in the same application. The use is quite limited, but I was curious about its filtering capability of it.

We implemented the notification system between our components, and we found that Apache Kafka performs well in scalability. It has improved our organization because of the scalability and the comfort of a fail-safe or disaster recovery it provides.

Apache Kafka can improve by making the documentation more user-friendly. It would be beneficial if we could explain to customers in more detail how the solution operates but the documentation get highly technical quickly. For example, if they had a simple page where we can show the customers how it works without the need for the customer to have a computer science background.

I have been using Apache Kafka for approximately two years.

Apache Kafka is scalable. It is easy to add brokers.

We have approximately 30 people using this solution in my organization. They use the solution daily.

I have only used Apache Kafka.

The initial setup of Apache Kafka took some time but after it was easy.

I rate the initial setup of Apache Kafka a three out of five.

We set up the solution in-house.

This is an open-source solution and is free to use.

We have not used the solution in production. We do not have a lot of data at the moment.

I would recommend this solution to others.

I rate Apache Kafka an eight out of ten.