Apache Kafka Reviews

The most valuable features are topic-based eventing, scalability, and retention periods.

My organization is transforming by using the new SOA/eventing-based architecture. The application depends on the employees’ information events. Kafka is very helpful in implementing this. It increases the performance and gives the details to multiple external/internal teams using Kafka topics in an asynchronous manner.

For example, if someone is moving from one office to another one, we have to update the software. While updating it, the system puts that event in a topic so that all other consumers can update that person’s new location. This can include the payroll team, the insurance team, and the hospital network.

The retention period helps us retain the data in the topic for the configured number of days. In this example, if any of the consumers fail to consume the message from the topic, then that message will be there until the retention period ends.

I would like to see a more user-friendly GUI.

We have used this solution since December, 2015.

If you are using the same group ID for multiple topics, it may shut down the application. We have faced this issue before.

We have not had any scalability issues.

I would give technical support a rating of 6 out of 10.

We were using ActiveMQ, which is just a messaging system. We are changing because of Kafka’s added value of scalability, retention, and high payload support.

The installation was somewhat straightforward.

The solution is worth the money.

This is the best tool I have ever used for asynchronous, event-based solutions.

• Supports more than 10,000 events/second.
• Scalability
• Replication

It is a good product for event-driven architecture.

We use Kafka for reactive architecture, track and trace, mail and parcel.

A good free monitor tool would be great for Apache Kafka (from Apache foundation).

We used Kafka 0.8 for 2 years and Kafka 0.10 for 3 months.

We have not encountered any stability issues.

We have not encountered any scalability issues.

We haven’t used technical support.

Apache MQ is different. It is a message bus (log rotate) than can manage more than 10,000 events/sec.

The basic configuration is quite good. We have built a Hadoop cluster and the Kafka service was included.

We use a community version.

Kafka processes asynchronous exchanges, so there are no transactional interactions.

I am a user, as well as an integrator for our clients. This is one of the products that we implement for others.

The most valuable features of this solution is the architectural style of messaging or event streaming. First is important to understand that anything can be represented as an event=message e.g new order, status change, confirmation, information from IoT or from monitoring system. Each message can be transport very quickly from the source (producer) into consumer(s). Messages can be changed in the fly - streaming messaging.

Messages can be process exactly at once or at least once. Ordering of messages depends on just configuration setup. 

When comparing it with other messaging and integration platforms, this is one of the best rated. Message store is out of the box functionality. Messages are automatically stored based on parameter setup of retention policy. Messages can remain for longer, which can be configured from a few milliseconds up to years. Scalability and availability of messages can be changed with zero maintenance window.

You don't need extra clusters to achieve high availability for the messaging system like Veritas, PowerHA or other.

One platform for classical messaging, real time messaging, ETL, message streaming. 

ETL can be realised with connectors into external systems which also could run in more instances. Exists a lot of ready to use connectors and new ones can be developed.

The model where you create the integration or the integration scenario needs improvement. It contains fewer developer words or maintaining words where someone prepared the topics, the connectors, or the streaming platforms. You would first need to have a control center from a third party for managing. 

If you would like to prepare something that is a more sophisticated integration scenario, where you use one microservice to provide the event or a second to several that consumed these microservices, then this needs to be modeled elsewhere. 

Also, when comparing to the traditional ESD for data mixing, you can create a scenario that could be deployed with inputs and some outputs.

Most business like the topics, but for me, I think that it is a problem that messaging platforms have, there is no design tool with IDE for creating.

It would be helpful to create a more complex solution for several types of styles, and not just for one provider or for one customer. That would be easier, but if you have more than one consumer then it could be a more complex scenario. It would be like events that go to several microservers to create orders, validate orders, and creating words. This would be helpful.

In the next release, adding some IDE or developing tools, for creating better integration scenarios, even though it already a developer-oriented solution, would be helpful. It would also be helpful for the auto-deployment.

Having a governance style would also be helpful to understand. 

It would be beneficial to have a repository of all of the topics, data types that exist, or data structures.

I have been working with this solution for one year.

It's a stable solution.

Kafka is very scalable, which is an important feature of it.

Our clients have approximately ten applications in their companies that communicate with Kafka.

I have not contacted technical support through Kafka, I communicate directly with Confluence. Confluence is the company that developed the open-source platform and they provide support.

The communication is very good and they are very capable of assisting you with all technical inquiries.

There are direct contacts that make it easy.

Previously, I worked with IBM MQ, a different type of messaging platform.

For the most part, the initial setup is easy, but if you need a more sophisticated infrastructure or if you have to set up the topics, then you have to be careful and you have to be more knowledgable in Kafka. You will have to know the parameters for the rotations, the size of the message, and the timeouts, as an example.

For a developer it is easy, but for an administrator and production, it requires more.

Apache Kafka is an open-source solution and there are no fees, but there are fees associated with confluence, which are based on subscription.

I would recommend trying this solution. 

Take the time to understand it because it is a different style when it comes to working with data.

I would rate this solution a nine out of ten.

We are currently using this solution on our cloud-based clusters.

We use Kafka as part of our services. Our product (cloud clusters) has many components and Kafka is one of them.

For example, we use Kafka as a data integration tool. If you take Oracle GoldenGate as a typical use case, what happens is GoldenGate collects the data for the replication and sends this data to the Kafka servers. We collect the data on the Kafka servers, and we create some transformations, some operations, from that data. We then copy the data to the HTTP or hub site.

Previously, when I worked at Nokia, we were collected data using Kafka and then we stored the data on the Kafka servers. We did all transformations through Kafka streaming. Later, Kafka moved data over to the HP site. 

Kafka has a good storage layer on its side. I can store this data if it's streaming, and, if we do encounter any error, for example, on the network or server, we can later use the data to do some analytics on it using the Kafka server.

Kafka provides us with a way to store the data used for analytics. That's the big selling point. There's very good log management. 

Kafka provides many APIs that can be flexible and can be placed or expanded using the development life cycle. For example, using Java, I can customize the API according to our customers' demands. I can expand the functionality according to our customer demands as well. It's also possible to create some models. It allows for more flexibility than much of the competition.

If the graphical user interface was easier for the Kafka administration it would be much better. Right now, you need to use the program with a command-line interface. If the graphical user interface was easier, it could be a better product.

I've been using the solution for more than three years.

The solution can be quite stable. We haven't encountered any issues on the Kafka side. However, Creating custom stabilizations would be good for dealing with stabilizing issues.

The scalability of the solution is very good. You can analyze system events horizontally and the cluster can be brought over to the cloud side with the Kafka user's server.

We use the solution for both small and medium-sized organizations, but also larger enterprises. Some of our clients are in the banking and financial sector.

Officially, I did not create any Kafka support tasks on the configuration support that is offered. I have created some questions on the stack overflow, however. Technical support is very good and I've found their response is very quick, giving you an answer within a day.

We didn't previously use a different solution. We did some applications with Java for the consumer content but not the application function within that. We did objects instead.

The initial setup isn't too complex. I know Kafka very well and don't find it to be overly difficult. There's also very good documentation which users can take advantage of.

Deployment, including security integration, only took about one day.

Two people handled the deployment. One person created the authentification group and after creating groups and users, another handled topic authentification and user definition for the customer.

I handled the implementation for cloud-based clusters. I defined the broker nodes and other nodes for Kafka. We are a cloud integrator, so we handled it ourselves.

I'm unaware of the costs surrounding licensing and setup.

We're using the 2.1.30 version of the solution for our cloud-based clusters. We use the on-premises deployment model. Most customers use the on-premise solution for cloud-based clusters.

Kafka is a very good solution for log management. If you need anything done related to log management, Kafka can do it. Kafka can also store the data in the brokers. This prevents data loss as well as the duplication of data. It's quite comprehensive.

I'd rate the solution seven out of ten. If the solution could provide a user interface I'd rate it higher. This is important for managing Kafka's clusters on the administration side. It would also be helpful if two to three files could be minimized to one configuration file.

I like Kafka's flexibility, stability, reliability, and robustness.

Kafka has a lot of monitors, but sometimes it's most important to just have a simple monitor. Improvements to Kafka's management would be nice, but it's not so necessary for me. There are a lot of consoles that offer a better view than Kafka. Some are free, and some are paid, but I'm thinking about streaming. For example, if you connect more streams to a component in the same queue, how will it integrate to recognize the flow and the message?

I've been using Kafka for more than two years.

Kafka is stable. Defining our user base is hard because Kafka influences the whole company, so you could say around 100 users. Kafka is a core system, so it affects all users we choose to link to the primary key.

I previously used IBM MQ. 

The first time we tried to deploy Kafka, it seemed a little complicated, but the second try went better. Sometimes it isn't easy to set up the necessary communication or estimate how many partitions we need. Some applications have a vast amount of data, so we have to consider how to improve the performance and not increase the transfer times.

I rate Apache Kafka nine out of 10. I think it's one of the best tools on the internet.

• Distributed
• Persistence
• Offset management by consumer

This is the base streaming component of our IoT platform.

In case of disaster recovery, we mirror the data in the cluster by maintaining the offsets and store the data within Hadoop 2.8 HDFS.

• It needs a separate cluster and a separate administrator to manage the Kafka cluster, adding an extra cost.
• It is challenging when data is moved to a mirror cluster, in the case of disaster recovery. It doesn't keep the offset.

I have used this solution for one year.

The open source community is very strong. Also, distributors like Cloudera and Hortonworks provide paid support.

For big data, we did not have a previous solution. I have used Microsoft MQ for building traditional systems.

The setup was straightforward.

This is open source with the cost of a cluster administrator.

We did not look at anything else. At that time, this was already accepted by the industry for streaming data processing.

If the Hadoop distribution is MapR, then consider MapR Streaming. MapR Streaming has overcome these fundamental issues. It stores data within the MapR-FS itself. So there is extra overhead, but with a licensing cost.

We have used Kafka for streaming customer web clicks from live sessions to understand customer behavioral patterns.

The ability to partition data on Kafka is valuable. But Kafka needs support and management. It is better to have it fully managed on the cloud.

The only reason I give Kafka as product a low rating is because there are far superior and cheaper alternatives in cloud-based solutions, where we save money on manpower, electricity, servers, datacenters, networking, etc.

In fact, this is the view I have for pretty much all open source software compared to cloud based services. They just make things cheaper, faster, scalable and manageable. Kafka is good, but Kafka as a cloud service is awesome!!

This is a relative rating (compared to cloud services), not that something is wrong with Kafka. I hope that is clear.

The product is good, but it needs implementation and on-going support. The whole cloud engagement model has made the adoption of Kafka better due to PaaS (Amazon Kinesis, a fully managed service by AWS).

No issues here with stability.

Ah, scalability!!! We need to set up multiple servers again for handling the load, which makes Kafka not scalable, unless you subscribe to cloud services.

It’s an Apache-community based support, so it is not really prioritized if you have a business issue. This is why most enterprise customers pay for cloud services.

We didn’t have a previous solution. We started with Kafka and then switched to Amazon Kinesis (PaaS for Kafka). I think Microsoft Azure also released a competing service.

The setup was straightforward.

Licensing issues are not applicable. Apache licensing makes it simple with almost zero cost for the software itself.

We unsuccessfully, and kind of foolishly, tried Apache Camel. They were not similar in services, so we moved to Kafka rightfully, and then to AWS cloud ultimately.

If you have a dedicated Kafka resource to implement and manage the services, then go for Apache Kafka. Otherwise, do consider cloud-based services from AWS or Azure.

Apache Kafka is used for stream processing, metric and log aggregations, and as a message queue for connecting different microservices.

The stream processing is a very valuable aspect of the solution for us.

Due to the fact that the solution is open source, it has a zookeeper dependency. If I could change anything about the solution, it would be that.

The solution could always add a few more features to enhance its usage.

I've been with the company for at least one year, which is for how long I've been using the solution.

The stability of the solution is very good, even for large enterprise-level organizations. It's quite reliable. There aren't bugs or glitches that affect it. The solution doesn't crash.

The solution is scalable, however, it's a 50/50 endeavor. It may require some management to build it out.

The solution is open source, so there isn't technical support per se. The open-source community that surrounds the technology, however, is very good.

That said, our company provides technical support to our clients if they need it. It's 24/7 support and we try to reply within 20 minutes of receiving a request.

Some of our clients are using Apache, while others are using other solutions. It depends on the company and its unique requirements.

The difficulty or simplicity of the initial setup varies. It really depends on the organization and its requirements and infrastructure.

Deployment times vary. It can be up to a week in production, however, with some products online, some services can be deployed within minutes.

When you have already deployed the solution, and it's installed, it doesn't require very much maintenance. If it needs any, my company handles it for our clients. We have an entire team that can work on it.

The solution is open source; it's free to use.

What happens in our company is a little different. We basically provide services to other companies through Kafka, like our management services. It doesn't necessarily mean we're using the solution ourselves, however, we will be going and deploying Kafka for companies, like a systems integrator.

The version of the solution is normally 2.4, however, it depends on the requirements. Our cloud providers are always different due to the fact that the countries that we work with are all different. For example, in the US it could Amazon, Azure, or Google. It varies.

I'd advise other organizations considering using the solution to make sure they understand what the use case is. They need to know what their services will be and if they will be directed to Apache Kafka.

From a customer perspective, potential companies need to make sure they have an idea of how big it's going to be due to the fact that it's a cluster environment. It needs to be taken care of. Customers will need to know things like what is the message rate is which is coming into Kafka and how they will connect all those different microservices or any services together to Kafka.

From an infrastructure perspective, it's more of how big of a cluster a company needs. Who would be the producers to produce it, and who's the consumer who's consuming the data are a few questions that need to be asked.

I'd rate the solution eight out of ten.