Amazon MSK vs Databricks comparison

Read 82 Databricks reviews

7,736 Views
5,084 Comparison Views

96% willing to recommend

Amazon MSK

Comparison Buyer's Guide

Download the report

Executive Summary

We performed a comparison between Amazon MSK and Databricks based on real PeerSpot user reviews.

Find out in this report how the two Streaming Analytics solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.

To learn more, read our detailed Amazon MSK vs. Databricks Report (Updated: October 2024).

Amazon MSK vs. Databricks

October 2024

Download the complete report

Helped 814,763 peers since 2012

Categories and Ranking

Amazon MSK

Ranking in Streaming Analytics

6th

Average Rating

7.4

Number of Reviews

Ranking in other categories

No ranking in other categories

Databricks

Ranking in Streaming Analytics

1st

Average Rating

8.2

Number of Reviews

Ranking in other categories

Data Science Platforms (1st)

Mindshare comparison

As of November 2024, in the Streaming Analytics category, the mindshare of Amazon MSK is 9.2%, down from 9.9% compared to the previous year. The mindshare of Databricks is 14.0%, up from 9.6% compared to the previous year. It is calculated based on PeerSpot user engagement data.

Streaming Analytics

Featured Reviews

FNU AKSHANSH

Senior Data Engineer at a computer software company with 201-500 employees

May 9, 2024

Streamlines our processes, and we don't need to configure any VPCs; it's automatic

Amazon MSK has good integration because our team has been undergoing significant changes. Coupling it with MSK within AWS is helpful. We don't have to set up additionals or monitor external environments. This coupling streamlines our processes, and we don't need to configure any VPCs; it's…

Read full review

Dunstan Matekenya

Data Scientist at a financial services firm with 10,001+ employees

Jul 10, 2024

Process large-scale data sets and integrates with Apache Spark with notebook environment

I primarily use Databricks to process large-scale data sets with Apache Spark. My main use case is processing large data sets, such as 600 GB or 800 GB Databricks integrates natively with Apache Spark, which I use as a processing engine for large-scale datasets. This native integration is one of…

Read full review

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros

"It offers good stability."

"Amazon MSK has good integration because our team has been undergoing significant changes. Coupling it with MSK within AWS is helpful. We don't have to set up additionals or monitor external environments. This"

"It is a stable product."

"Amazon MSK's scalability is very good."

"MSK has a private network that's an out-of-box feature."

"Overall, it is very cost-effective based on the workflow."

"Amazon MSK's separation of concerns and ease of creating and deploying new features are highly valuable. It just requires to assign them to the topic, and then anyone who needs to consume these messages can do so directly from Amazon MSK. This separation of concerns makes it very convenient, especially for new feature development, as developers can easily access the messages they need without having to deal with complex server communications or protocol setups."

"The most valuable feature of Amazon MSK is the integration."

More Amazon MSK pros

"It can send out large data amounts."

"The capacity of use of the different types of coding is valuable. Databricks also has good performance because it is running in spark extra storage, meaning the performance and the capacity use different kinds of codes."

"Databricks helps crunch petabytes of data in a very short period of time."

"The technical support is good."

"Databricks integrates well with other solutions."

"Imageflow is a visual tool that helps make it easier for business people to understand complex workflows."

"The most valuable features of the solution are the hardware and the resources it quickly provides without much hassle."

"We like that this solution can handle a wide variety and velocity of data engineering, either in batch mode or real-time."

More Databricks pros

Cons

"One of the reasons why we prefer Kafka is because the support is a little bit difficult to manage with Amazon MSK."

"The product's schema support needs enhancement. It will help enhance integration with many kinds of languages of programming languages, especially for environments using languages like .NET."

"It would be really helpful if Amazon MSK could provide a single installation that covers all the servers."

"It should be more flexible, integration-wise."

"The configuration seems a little complex and the documentation on the product is not available."

"Amazon MSK could improve on the features they offer. They are still lagging behind Confluence."

"Horizontal scale-out is actually not easy, making it an area where improvements are required."

"It does not autoscale. Because if you do keep it manually when you add a note to the cluster and then you register it, then it is scalable, but the fact that you have to go and do it, I think, makes it, again, a bit of some operational overhead when managing the cluster."

More Amazon MSK cons

"I'm not the guy that I'm working with Databricks on a daily basis. I'm on the management team. However, my team tells me there are limitations with streaming events. The connectors work with a small set of platforms. For example, we can work with Kafka, but if we want to move to an event-driven solution from AWS, we cannot do it. We cannot connect to all the streaming analytics platforms, so we are limited in choosing the best one."

"The product needs samples and templates to help invite users to see results and understand what the product can do."

"Instead of relying on a massive instance, the solution should offer micro partition levels. They're working on it, however, they need to implement it to help the solution run more effectively."

"The product should provide more advanced features in future releases."

"Databricks' technical support takes a while to respond and could be improved."

"The tool should improve its integration with other products."

"The data visualization for this solution could be improved. They have started to roll out a data visualization tool inside Databricks but it is in the early stages. It's not comparable to a solution like Power BI, Luca, or Tableau."

"Some of the error messages that we receive are too vague, saying things like "unknown exception", and these should be improved to make it easier for developers to debug problems."

More Databricks cons

Pricing and Cost Advice

"The platform has better pricing than one of its competitors."

"The price of Amazon MSK is less than some competitor solutions, such as Confluence."

"When you create a complete enterprise-driven architecture that is deployable on an enterprise scale, I would say that the prices of Amazon MSK and Confluent Platform become comparable."

"The solution uses a pay-per-use model with an annual subscription fee or package. Typically this solution is used on a cloud platform, such as Azure or AWS, but more people are choosing Azure because the price is more reasonable."

"We find Databricks to be very expensive, although this improved when we found out how to shut it down at night."

"Licensing on site I would counsel against, as on-site hardware issues tend to really delay and slow down delivery."

"The licensing costs of Databricks is a tiered licensing regime, so it is flexible."

"The cost for Databricks depends on the use case. I work on it as a consultant, so I'm using the client's Databricks, so it depends on how big the client is."

"The pricing depends on the usage itself."

"We have only incurred the cost of our AWS cloud services. This is because during this period, Databricks provided us with an extended evaluation period, and we have not spent much money yet. We are just starting to incur costs this month, I will know more later on the full cost perspective."

"Databricks' cost could be improved."

More Databricks pricing and cost advice

See which vendors are best for you

Use our free recommendation engine to learn which Streaming Analytics solutions are best for your needs.

See recommendations

814,763 professionals have used our research since 2012.

Top Industries

By visitors reading reviews

Financial Services Firm

21%

Computer Software Company

19%

Manufacturing Company

Retailer

Financial Services Firm

16%

Computer Software Company

12%

Manufacturing Company

Healthcare Company

Company Size

By reviewers

Large Enterprise

Midsize Enterprise

Small Business

Questions from the Community

What do you like most about Amazon MSK?

Amazon MSK has significantly improved our organization by building seamless integration between systems.

What needs improvement with Amazon MSK?

From AWS, I would consider more MSK schema validation is needed. It is easy to integrate if you have an application, but on-topic integration is more complex. You can do it with EvenBridge very eas...

What is your primary use case for Amazon MSK?

I have used Confluent Cloud and Amazon MSK in my company. We are not using it for analytics and it is more for CDC processes, so we change the capture processes. It is used to extract data from a d...

Which do you prefer - Databricks or Azure Machine Learning Studio?

Databricks gives you the option of working with several different languages, such as SQL, R, Scala, Apache Spark, or Python. It offers many different cluster choices and excellent integration with ...

How would you compare Databricks vs Amazon SageMaker?

We researched AWS SageMaker, but in the end, we chose Databricks. Databricks is a Unified Analytics Platform designed to accelerate innovation projects. It is based on Spark so it is very fast. It...

Which would you choose - Databricks or Azure Stream Analytics?

Databricks is an easy-to-set-up and versatile tool for data management, analysis, and business analytics. For analytics teams that have to interpret data to further the business goals of their orga...

Azure Stream Analytics vs Amazon MSK

Comparisons

Compared 29% of the time

Confluent vs Amazon MSK

Compared 24% of the time

Amazon Kinesis vs Amazon MSK

Compared 17% of the time

Google Cloud Dataflow vs Amazon MSK

Compared 13% of the time

Apache Flink vs Amazon MSK

Compared 5% of the time

More Amazon MSK Competitors

Dataiku vs Databricks

Compared 11% of the time

Informatica PowerCenter vs Databricks

Compared 8% of the time

Amazon SageMaker vs Databricks

Compared 8% of the time

Dremio vs Databricks

Compared 8% of the time

Microsoft Power BI vs Databricks

Compared 6% of the time

More Databricks Competitors

Product Reports

Download Amazon MSK product report

Amazon MSK

November 2024

Download Databricks product report

November 2024

Also Known As

Amazon Managed Streaming for Apache Kafka

Databricks Unified Analytics, Databricks Unified Analytics Platform, Redash

Learn More

Amazon Web Services (AWS)

Overview

Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed service that enables you to build and run applications that use Apache Kafka to process streaming data. Amazon MSK provides the control-plane operations, such as those for creating, updating, and deleting clusters.

Databricks is utilized for advanced analytics, big data processing, machine learning models, ETL operations, data engineering, streaming analytics, and integrating multiple data sources.

Organizations leverage Databricks for predictive analysis, data pipelines, data science, and unifying data architectures. It is also used for consulting projects, financial reporting, and creating APIs. Industries like insurance, retail, manufacturing, and pharmaceuticals use Databricks for data management and analytics due to its user-friendly interface, built-in machine learning libraries, support for multiple programming languages, scalability, and fast processing.

What are the key features of Databricks?

User-friendly interface: Simplifies operations and usability.
Built-in machine learning libraries: Facilitates machine learning tasks.
Support for multiple programming languages: Enhances flexibility.
Scalability: Efficiently handles growing data needs.
Fast processing: Improves performance and speed.
Automated optimization: Reduces manual efforts.
Data visualization: Provides insightful visuals.
Collaborative features: Enhances teamwork.
Delta Lake performance: Boosts data management.
Seamless cluster management: Simplifies system operations.

What are the benefits or ROI to look for in Databricks reviews?

Efficiency in handling large datasets: Ensures smooth processing.
Interactive workspace environment: Improves user collaboration.
Integration with platforms: Provides connectivity benefits.
Performance optimization: Enhances overall system performance.
Support for data governance and security: Ensures data integrity and protection.

Databricks is implemented in insurance for risk analysis and claims processing; in retail for customer analytics and inventory management; in manufacturing for predictive maintenance and supply chain optimization; and in pharmaceuticals for drug discovery and patient data analysis. Users value its scalability, machine learning support, collaboration tools, and Delta Lake performance but seek improvements in visualization, pricing, and integration with BI tools.

Sample Customers

Expedia, Intuit, Royal Dutch Shell, Brooks Brothers

Elsevier, MyFitnessPal, Sharethrough, Automatic Labs, Celtra, Radius Intelligence, Yesware