Try our new research platform with insights from 80,000+ expert users

Amazon MSK vs Google Cloud Dataflow comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Dec 17, 2024

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Categories and Ranking

Amazon MSK
Ranking in Streaming Analytics
6th
Average Rating
7.4
Reviews Sentiment
7.1
Number of Reviews
10
Ranking in other categories
No ranking in other categories
Google Cloud Dataflow
Ranking in Streaming Analytics
8th
Average Rating
7.8
Reviews Sentiment
7.3
Number of Reviews
10
Ranking in other categories
No ranking in other categories
 

Mindshare comparison

As of January 2025, in the Streaming Analytics category, the mindshare of Amazon MSK is 8.9%, down from 10.0% compared to the previous year. The mindshare of Google Cloud Dataflow is 8.4%, up from 7.0% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Streaming Analytics
 

Featured Reviews

FNU AKSHANSH - PeerSpot reviewer
Streamlines our processes, and we don't need to configure any VPCs; it's automatic
We don't have many use cases involving ingesting large amounts of data and scaling up and down. We have a clear understanding of our data volume, which remains relatively constant throughout the week. While we're aware of other features Amazon MSK offers, we feel confident in our current setup. If our requirements change significantly in the future, we'll reassess our needs and consider adopting Amazon MSK.
Tamer Talal - PeerSpot reviewer
A tool useful for data transmission and data storage that needs to improve its authentication area
The authentication part of the product is an area of concern where improvements are required. For some common users, the solution's authentication part is difficult to use. The scalability of the product is an area of concern where improvements are required. In the future, the product should be made available at a cheaper rate.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"Amazon MSK has significantly improved our organization by building seamless integration between systems."
"The most valuable feature of Amazon MSK is the integration."
"Amazon MSK's scalability is very good."
"MSK has a private network that's an out-of-box feature."
"It offers good stability."
"The solution's technical support was helpful."
"It is a stable product."
"Amazon MSK's separation of concerns and ease of creating and deploying new features are highly valuable. It just requires to assign them to the topic, and then anyone who needs to consume these messages can do so directly from Amazon MSK. This separation of concerns makes it very convenient, especially for new feature development, as developers can easily access the messages they need without having to deal with complex server communications or protocol setups."
"The best feature of Google Cloud Dataflow is its practical connectedness."
"The most valuable features of Google Cloud Dataflow are scalability and connectivity."
"Google Cloud Dataflow is useful for streaming and data pipelines."
"I don't need a server running all the time while using the tool. It is also easy to setup. The product offers a pay-as-you-go service."
"The product's installation process is easy...The tool's maintenance part is somewhat easy."
"The most valuable features of Google Cloud Dataflow are the integration, it's very simple if you have the complete stack, which we are using. It is overall very easy to use, user-friendly friendly, and cost-effective if you know how to use it. The solution is very flexible for programmers, if you know how to do scripts or program in Python or any other language, it's extremely easy to use."
"It is a scalable solution."
"The service is relatively cheap compared to other batch-processing engines."
 

Cons

"The configuration seems a little complex and the documentation on the product is not available."
"In my opinion, there are areas in Amazon MSK that could be improved, particularly in terms of configuration. Initially setting it up and getting it connected was quite challenging. The naming conventions for policies were updated by AWS, and some were undocumented, leading to confusion with outdated materials. It took us weeks of trial and error before discovering new methods through hidden tutorials and official documentation."
"It would be really helpful if Amazon MSK could provide a single installation that covers all the servers."
"Horizontal scale-out is actually not easy, making it an area where improvements are required."
"It should be more flexible, integration-wise."
"It does not autoscale. Because if you do keep it manually when you add a note to the cluster and then you register it, then it is scalable, but the fact that you have to go and do it, I think, makes it, again, a bit of some operational overhead when managing the cluster."
"One of the reasons why we prefer Kafka is because the support is a little bit difficult to manage with Amazon MSK."
"Amazon MSK could improve on the features they offer. They are still lagging behind Confluence."
"The deployment time could also be reduced."
"Google Cloud Dataflow should include a little cost optimization."
"The authentication part of the product is an area of concern where improvements are required."
"There are certain challenges regarding the Google Cloud Composer which can be improved."
"The solution's setup process could be more accessible."
"They should do a market survey and then make improvements."
"When I deploy the product in local errors, a lot of errors pop up which are not always caught. The solution's error logging is bad. It can take a lot of time to debug the errors. It needs to have better logs."
"Google Cloud Data Flow can improve by having full simple integration with Kafka topics. It's not that complicated, but it could improve a bit. The UI is easy to use but the experience could be better. There are other tools available that do a better job."
 

Pricing and Cost Advice

"When you create a complete enterprise-driven architecture that is deployable on an enterprise scale, I would say that the prices of Amazon MSK and Confluent Platform become comparable."
"The platform has better pricing than one of its competitors."
"The price of Amazon MSK is less than some competitor solutions, such as Confluence."
"On a scale from one to ten, where one is cheap, and ten is expensive, I rate the solution's pricing a seven to eight out of ten."
"Google Cloud is slightly cheaper than AWS."
"Google Cloud Dataflow is a cheap solution."
"The price of the solution depends on many factors, such as how they pay for tools in the company and its size."
"On a scale from one to ten, where one is cheap, and ten is expensive, I rate Google Cloud Dataflow's pricing a four out of ten."
"The solution is not very expensive."
"The tool is cheap."
"The solution is cost-effective."
report
Use our free recommendation engine to learn which Streaming Analytics solutions are best for your needs.
831,265 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
22%
Computer Software Company
19%
Manufacturing Company
6%
Retailer
6%
Financial Services Firm
17%
Retailer
13%
Manufacturing Company
12%
Computer Software Company
11%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about Amazon MSK?
Amazon MSK has significantly improved our organization by building seamless integration between systems.
What needs improvement with Amazon MSK?
From AWS, I would consider more MSK schema validation is needed. It is easy to integrate if you have an application, but on-topic integration is more complex. You can do it with EvenBridge very eas...
What is your primary use case for Amazon MSK?
I have used Confluent Cloud and Amazon MSK in my company. We are not using it for analytics and it is more for CDC processes, so we change the capture processes. It is used to extract data from a d...
What do you like most about Google Cloud Dataflow?
The product's installation process is easy...The tool's maintenance part is somewhat easy.
What needs improvement with Google Cloud Dataflow?
The authentication part of the product is an area of concern where improvements are required. For some common users, the solution's authentication part is difficult to use. The scalability of the p...
 

Also Known As

Amazon Managed Streaming for Apache Kafka
Google Dataflow
 

Overview

 

Sample Customers

Expedia, Intuit, Royal Dutch Shell, Brooks Brothers
Absolutdata, Backflip Studios, Bluecore, Claritics, Crystalloids, Energyworx, GenieConnect, Leanplum, Nomanini, Redbus, Streak, TabTale
Find out what your peers are saying about Amazon MSK vs. Google Cloud Dataflow and other solutions. Updated: January 2025.
831,265 professionals have used our research since 2012.