Try our new research platform with insights from 80,000+ expert users

Amazon MSK vs Databricks comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Dec 17, 2024

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Categories and Ranking

Amazon MSK
Ranking in Streaming Analytics
6th
Average Rating
7.4
Reviews Sentiment
7.1
Number of Reviews
10
Ranking in other categories
No ranking in other categories
Databricks
Ranking in Streaming Analytics
1st
Average Rating
8.2
Reviews Sentiment
7.0
Number of Reviews
85
Ranking in other categories
Data Science Platforms (1st)
 

Mindshare comparison

As of January 2025, in the Streaming Analytics category, the mindshare of Amazon MSK is 8.9%, down from 10.0% compared to the previous year. The mindshare of Databricks is 14.6%, up from 10.1% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Streaming Analytics
 

Featured Reviews

FNU AKSHANSH - PeerSpot reviewer
Streamlines our processes, and we don't need to configure any VPCs; it's automatic
We don't have many use cases involving ingesting large amounts of data and scaling up and down. We have a clear understanding of our data volume, which remains relatively constant throughout the week. While we're aware of other features Amazon MSK offers, we feel confident in our current setup. If our requirements change significantly in the future, we'll reassess our needs and consider adopting Amazon MSK.
Parag Bhosale - PeerSpot reviewer
Integrating engineering and learning, but cost challenges arise with cluster management
We often use a single cluster to ingest Databricks, which Databricks doesn't recommend. They suggest using a no-cluster solution like job clusters. This can be overwhelming for us because we started smaller. We prefer using a small to mid-sized cluster for many jobs to keep costs low, but this sometimes doesn't support our operations properly. We need to stay in sync with the DVR versions, and migrations can pose challenges. For example, issues arose when we moved a cluster from a previous version to the latest one. We could use their job clusters, however, that increases costs, which is challenging for us as a startup. Maintaining this infrastructure can be a headache.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"It offers good stability."
"Overall, it is very cost-effective based on the workflow."
"Amazon MSK has good integration because our team has been undergoing significant changes. Coupling it with MSK within AWS is helpful. We don't have to set up additionals or monitor external environments. This"
"Amazon MSK has significantly improved our organization by building seamless integration between systems."
"Amazon MSK's separation of concerns and ease of creating and deploying new features are highly valuable. It just requires to assign them to the topic, and then anyone who needs to consume these messages can do so directly from Amazon MSK. This separation of concerns makes it very convenient, especially for new feature development, as developers can easily access the messages they need without having to deal with complex server communications or protocol setups."
"It is a stable product."
"The solution's technical support was helpful."
"Amazon MSK's scalability is very good."
"The simplicity of development is the most valuable feature."
"Databricks' most valuable feature is the data transformation through PySpark."
"The ability to stream data and the windowing feature are valuable."
"Databricks is a unified solution that we can use for streaming. It is supporting open source languages, which are cloud-agnostic. When I do database coding if any other tool has a similar language pack to Excel or SQL, I can use the same knowledge, limiting the need to learn new things. It supports a lot of Python libraries where I can use some very easily."
"The most valuable feature of Databricks is the notebook, data factory, and ease of use."
"We like that this solution can handle a wide variety and velocity of data engineering, either in batch mode or real-time."
"The initial setup is pretty easy."
"Specifically for data science and data analytics purposes, it can handle large amounts of data in less time. I can compare it with Teradata. If a job takes five hours with Teradata databases, Databricks can complete it in around three to three and a half hours."
 

Cons

"One of the reasons why we prefer Kafka is because the support is a little bit difficult to manage with Amazon MSK."
"Horizontal scale-out is actually not easy, making it an area where improvements are required."
"The configuration seems a little complex and the documentation on the product is not available."
"Amazon MSK could improve on the features they offer. They are still lagging behind Confluence."
"It would be really helpful if Amazon MSK could provide a single installation that covers all the servers."
"In my opinion, there are areas in Amazon MSK that could be improved, particularly in terms of configuration. Initially setting it up and getting it connected was quite challenging. The naming conventions for policies were updated by AWS, and some were undocumented, leading to confusion with outdated materials. It took us weeks of trial and error before discovering new methods through hidden tutorials and official documentation."
"It does not autoscale. Because if you do keep it manually when you add a note to the cluster and then you register it, then it is scalable, but the fact that you have to go and do it, I think, makes it, again, a bit of some operational overhead when managing the cluster."
"The product's schema support needs enhancement. It will help enhance integration with many kinds of languages of programming languages, especially for environments using languages like .NET."
"It would be nice to have more guidance on integrations with ETLs and other data quality tools."
"The ability to customize our own pipelines would enhance the product, similar to what's possible using ML files in Microsoft Azure DevOps."
"The solution could improve by providing better automation capabilities. For example, working together with more of a DevOps approach, such as continuous integration."
"Databricks' performance when serving the data to an analytics tool isn't as good as Snowflake's."
"There is room for improvement in visualization."
"Databricks could improve in some of its functionality."
"Costs can quickly add up if you don't plan for it."
"Databricks can improve by making the documentation better."
 

Pricing and Cost Advice

"The price of Amazon MSK is less than some competitor solutions, such as Confluence."
"When you create a complete enterprise-driven architecture that is deployable on an enterprise scale, I would say that the prices of Amazon MSK and Confluent Platform become comparable."
"The platform has better pricing than one of its competitors."
"Databricks are not costly when compared with other solutions' prices."
"We implement this solution on behalf of our customers who have their own Azure subscription and they pay for Databricks themselves. The pricing is more expensive if you have large volumes of data."
"There are different versions."
"The solution is based on a licensing model."
"I would rate Databricks' pricing seven out of ten."
"I do not exactly know the costs, but one of our clients pays between $100 USD and $200 USD monthly."
"The solution uses a pay-per-use model with an annual subscription fee or package. Typically this solution is used on a cloud platform, such as Azure or AWS, but more people are choosing Azure because the price is more reasonable."
"Price-wise, I would rate Databricks a three out of five."
report
Use our free recommendation engine to learn which Streaming Analytics solutions are best for your needs.
831,265 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
22%
Computer Software Company
19%
Manufacturing Company
6%
Retailer
6%
Financial Services Firm
17%
Computer Software Company
11%
Manufacturing Company
9%
Healthcare Company
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about Amazon MSK?
Amazon MSK has significantly improved our organization by building seamless integration between systems.
What needs improvement with Amazon MSK?
From AWS, I would consider more MSK schema validation is needed. It is easy to integrate if you have an application, but on-topic integration is more complex. You can do it with EvenBridge very eas...
What is your primary use case for Amazon MSK?
I have used Confluent Cloud and Amazon MSK in my company. We are not using it for analytics and it is more for CDC processes, so we change the capture processes. It is used to extract data from a d...
Which do you prefer - Databricks or Azure Machine Learning Studio?
Databricks gives you the option of working with several different languages, such as SQL, R, Scala, Apache Spark, or Python. It offers many different cluster choices and excellent integration with ...
How would you compare Databricks vs Amazon SageMaker?
We researched AWS SageMaker, but in the end, we chose Databricks. Databricks is a Unified Analytics Platform designed to accelerate innovation projects. It is based on Spark so it is very fast. It...
Which would you choose - Databricks or Azure Stream Analytics?
Databricks is an easy-to-set-up and versatile tool for data management, analysis, and business analytics. For analytics teams that have to interpret data to further the business goals of their orga...
 

Comparisons

 

Also Known As

Amazon Managed Streaming for Apache Kafka
Databricks Unified Analytics, Databricks Unified Analytics Platform, Redash
 

Overview

 

Sample Customers

Expedia, Intuit, Royal Dutch Shell, Brooks Brothers
Elsevier, MyFitnessPal, Sharethrough, Automatic Labs, Celtra, Radius Intelligence, Yesware
Find out what your peers are saying about Amazon MSK vs. Databricks and other solutions. Updated: January 2025.
831,265 professionals have used our research since 2012.