Try our new research platform with insights from 80,000+ expert users

Apache Spark Streaming vs Databricks comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Dec 17, 2024

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Categories and Ranking

Apache Spark Streaming
Ranking in Streaming Analytics
10th
Average Rating
8.0
Reviews Sentiment
7.4
Number of Reviews
11
Ranking in other categories
No ranking in other categories
Databricks
Ranking in Streaming Analytics
1st
Average Rating
8.2
Reviews Sentiment
7.0
Number of Reviews
88
Ranking in other categories
Data Science Platforms (1st)
 

Mindshare comparison

As of January 2025, in the Streaming Analytics category, the mindshare of Apache Spark Streaming is 3.4%, down from 4.3% compared to the previous year. The mindshare of Databricks is 14.6%, up from 10.1% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Streaming Analytics
 

Featured Reviews

AbhishekGupta - PeerSpot reviewer
Easy integration, beneficial auto-scaling, and good open-sourced support community
The service structure of Apache Spark Streaming can improve. There are a lot of issues with memory management and latency. There is no real-time analytics. We recommend it for the use cases where there is a five-second latency, but not for a millisecond, an IOT-based, or the detection anomaly-based. Flink as a service is much better. Apache Spark Streaming does not have auto-tuning. A customer needs to invest a lot, in terms of management and maintenance.
Parag Bhosale - PeerSpot reviewer
Integrating engineering and learning, but cost challenges arise with cluster management
We often use a single cluster to ingest Databricks, which Databricks doesn't recommend. They suggest using a no-cluster solution like job clusters. This can be overwhelming for us because we started smaller. We prefer using a small to mid-sized cluster for many jobs to keep costs low, but this sometimes doesn't support our operations properly. We need to stay in sync with the DVR versions, and migrations can pose challenges. For example, issues arose when we moved a cluster from a previous version to the latest one. We could use their job clusters, however, that increases costs, which is challenging for us as a startup. Maintaining this infrastructure can be a headache.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"As an open-source solution, using it is basically free."
"Apache Spark Streaming was straightforward in terms of maintenance. It was actively developed, and migrating from an older to a newer version was quite simple."
"Apache Spark Streaming's most valuable feature is near real-time analytics. The developers can build APIs easily for a code-steaming pipeline. The solutions have an ecosystem of integration with other stock services."
"The platform’s most valuable feature for processing real-time data is its ability to handle continuous data streams."
"Spark Streaming is critical, quite stable, full-featured, and scalable."
"It's the fastest solution on the market with low latency data on data transformations."
"Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows."
"The solution is better than average and some of the valuable features include efficiency and stability."
"Having one solution for everything, from data engineering to machine learning, is beneficial since everything comes under one hood."
"It can send out large data amounts."
"Ability to work collaboratively without having to worry about the infrastructure."
"We like that this solution can handle a wide variety and velocity of data engineering, either in batch mode or real-time."
"We have the ability to scale, collaborate and do machine learning."
"I haven't heard about any major stability issues. At this time I feel like it's stable."
"We can scale the product."
"The most valuable aspect of the solution is its notebook. It's quite convenient to use, both terms of the research and the development and also the final deployment, I can just declare the spark jobs by the load tables. It's quite convenient."
 

Cons

"It was resource-intensive, even for small-scale applications."
"The solution itself could be easier to use."
"We don't have enough experience to be judgmental about its flaws."
"The cost and load-related optimizations are areas where the tool lacks and needs improvement."
"Integrating event-level streaming capabilities could be beneficial."
"We would like to have the ability to do arbitrary stateful functions in Python."
"The debugging aspect could use some improvement."
"In terms of improvement, the UI could be better."
"The biggest problem associated with the product is that it is quite pricey."
"We often use a single cluster to ingest Databricks, which Databricks doesn't recommend. They suggest using a no-cluster solution like job clusters. This can be overwhelming for us because we started smaller."
"CI/CD needs additional leverage and support."
"I would like it if Databricks adopted an interface more like R Studio. When I create a data frame or a table, R Studio provides a preview of the data. In R Studio, I can see that it created a table with so many columns or rows. Then I can click on it and open a preview of that data."
"There could be more support for automated machine learning in the database. I would like to see more ways to do analysis so that the reporting is more understandable."
"Pricing is one of the things that could be improved."
"The data visualization for this solution could be improved. They have started to roll out a data visualization tool inside Databricks but it is in the early stages. It's not comparable to a solution like Power BI, Luca, or Tableau."
"The product should incorporate more learning aspects. It needs to have a free trial version that the team can practice."
 

Pricing and Cost Advice

"I was using the open-source community version, which was self-hosted."
"People pay for Apache Spark Streaming as a service."
"On a scale from one to ten, where one is expensive, or not cost-effective, and ten is cheap, I rate the price a seven."
"Spark is an affordable solution, especially considering its open-source nature."
"It is an expensive tool. The licensing model is a pay-as-you-go one."
"The product pricing is moderate."
"Whenever we want to find the actual costing, we have to send an email to Databricks, so having the information available on the internet would be helpful."
"The solution requires a subscription."
"Databricks' cost could be improved."
"I do not exactly know the costs, but one of our clients pays between $100 USD and $200 USD monthly."
"The price is okay. It's competitive."
"The basic version of this solution is now open-source, so there are no license costs involved. However, there is a charge for any advanced functionality and this can be quite expensive."
report
Use our free recommendation engine to learn which Streaming Analytics solutions are best for your needs.
831,997 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
26%
Computer Software Company
20%
Manufacturing Company
6%
University
5%
Financial Services Firm
17%
Computer Software Company
11%
Manufacturing Company
9%
Healthcare Company
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about Apache Spark Streaming?
Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows.
What needs improvement with Apache Spark Streaming?
We don't have enough experience to be judgmental about its flaws, as we've only used stable features like batch micro-batch. Integration poses no problem; however, I don't use some features and can...
What is your primary use case for Apache Spark Streaming?
We use Spark Streaming in a micro-batch region. It's not a full real-time system, but it offers high performance and low latency.
Which do you prefer - Databricks or Azure Machine Learning Studio?
Databricks gives you the option of working with several different languages, such as SQL, R, Scala, Apache Spark, or Python. It offers many different cluster choices and excellent integration with ...
How would you compare Databricks vs Amazon SageMaker?
We researched AWS SageMaker, but in the end, we chose Databricks. Databricks is a Unified Analytics Platform designed to accelerate innovation projects. It is based on Spark so it is very fast. It...
Which would you choose - Databricks or Azure Stream Analytics?
Databricks is an easy-to-set-up and versatile tool for data management, analysis, and business analytics. For analytics teams that have to interpret data to further the business goals of their orga...
 

Also Known As

Spark Streaming
Databricks Unified Analytics, Databricks Unified Analytics Platform, Redash
 

Overview

 

Sample Customers

UC Berkeley AMPLab, Amazon, Alibaba Taobao, Kenshoo, eBay Inc.
Elsevier, MyFitnessPal, Sharethrough, Automatic Labs, Celtra, Radius Intelligence, Yesware
Find out what your peers are saying about Apache Spark Streaming vs. Databricks and other solutions. Updated: January 2025.
831,997 professionals have used our research since 2012.