Apache Spark Streaming vs Databricks comparison

Read 82 Databricks reviews

7,736 Views
5,084 Comparison Views

96% willing to recommend

Apache Spark Streaming

Comparison Buyer's Guide

Download the report

Executive Summary

We performed a comparison between Apache Spark Streaming and Databricks based on real PeerSpot user reviews.

Find out in this report how the two Streaming Analytics solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.

To learn more, read our detailed Apache Spark Streaming vs. Databricks Report (Updated: October 2024).

Apache Spark Streaming vs. Databricks

October 2024

Download the complete report

Helped 814,763 peers since 2012

Categories and Ranking

Apache Spark Streaming

Ranking in Streaming Analytics

9th

Average Rating

8.0

Number of Reviews

Ranking in other categories

No ranking in other categories

Databricks

Ranking in Streaming Analytics

1st

Average Rating

8.2

Number of Reviews

Ranking in other categories

Data Science Platforms (1st)

Mindshare comparison

As of November 2024, in the Streaming Analytics category, the mindshare of Apache Spark Streaming is 3.7%, down from 4.7% compared to the previous year. The mindshare of Databricks is 14.0%, up from 9.6% compared to the previous year. It is calculated based on PeerSpot user engagement data.

Streaming Analytics

Featured Reviews

Oscar Estorach

Chief Data-strategist and Director at Theworkshop.es

Jan 25, 2024

Versatile and flexible when dealing with large-scale data streams

What I like about Spark is its versatility in supporting multiple languages and that makes it my preferred choice for building scalable and efficient systems, whether it is hooking databases with web applications or handling large-scale data transformations. Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows. It works well in the cloud, and you can structure data using Databricks or Spark, providing flexibility for different projects. Spark Streaming's flexibility shines when dealing with large-scale data streams. It caters to different needs, offering real-time insights for tasks like online sales analytics. The ability to prioritize data streams is valuable, especially for monitoring competitor prices online.

Read full review

Dunstan Matekenya

Data Scientist at a financial services firm with 10,001+ employees

Jul 10, 2024

Process large-scale data sets and integrates with Apache Spark with notebook environment

I primarily use Databricks to process large-scale data sets with Apache Spark. My main use case is processing large data sets, such as 600 GB or 800 GB Databricks integrates natively with Apache Spark, which I use as a processing engine for large-scale datasets. This native integration is one of…

Read full review

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros

"Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows."

"As an open-source solution, using it is basically free."

"Apache Spark's capabilities for machine learning are quite extensive and can be used in a low-code way."

"It's the fastest solution on the market with low latency data on data transformations."

"Apache Spark Streaming has features like checkpointing and Streaming API that are useful."

"The solution is better than average and some of the valuable features include efficiency and stability."

"Apache Spark Streaming was straightforward in terms of maintenance. It was actively developed, and migrating from an older to a newer version was quite simple."

"The solution is very stable and reliable."

More Apache Spark Streaming pros

"The most valuable feature of Databricks is the integration with Microsoft Azure."

"Imageflow is a visual tool that helps make it easier for business people to understand complex workflows."

"Databricks is hosted on the cloud. It is very easy to collaborate with other team members who are working on it. It is production-ready code, and scheduling the jobs is easy."

"Databricks is a robust solution for big data processing, offering flexibility and powerful features."

"What I like about Databricks is that it's one of the most popular platforms that give access to folks who are trying not just to do exploratory work on the data but also go ahead and build advanced modeling and machine learning on top of that."

"Databricks has improved my organization by allowing us to transform data from sources to a different format and feed that to the analytics, business intelligence, and reporting teams. This tool makes it easy to do those kinds of things."

"Databricks' most valuable feature is the data transformation through PySpark."

"The most valuable features of the solution are the hardware and the resources it quickly provides without much hassle."

More Databricks pros

Cons

"There could be an improvement in the area of the user configuration section, it should be less developer-focused and more business user-focused."

"We would like to have the ability to do arbitrary stateful functions in Python."

"The debugging aspect could use some improvement."

"The initial setup is quite complex."

"The solution itself could be easier to use."

"In terms of improvement, the UI could be better."

"The cost and load-related optimizations are areas where the tool lacks and needs improvement."

"Integrating event-level streaming capabilities could be beneficial."

More Apache Spark Streaming cons

"It's not easy to use, and they need a better UI."

"I would love an integration in my desktop IDE. For now, I have to code on their webpage."

"I would like to see more documentation in terms of how an end-user could use it, and users like me can easily try it and implement use cases."

"The stability of the clusters or the instances of Databricks would be better if it was a much more stable environment. We've had issues with crashes."

"The solution has some scalability and integration limitations when consolidating legacy systems."

"A lot of people are required to manage this solution."

"This solution only supports queries in SQL and Python, which is a bit limiting."

"When I used the support, I had communication problems because of the language barrier with the agent. The accent was difficult to understand."

More Databricks cons

Pricing and Cost Advice

"People pay for Apache Spark Streaming as a service."

"On a scale from one to ten, where one is expensive, or not cost-effective, and ten is cheap, I rate the price a seven."

"I was using the open-source community version, which was self-hosted."

"Spark is an affordable solution, especially considering its open-source nature."

"I do not exactly know the costs, but one of our clients pays between $100 USD and $200 USD monthly."

"The solution is affordable."

"The solution requires a subscription."

"The licensing costs of Databricks is a tiered licensing regime, so it is flexible."

"We implement this solution on behalf of our customers who have their own Azure subscription and they pay for Databricks themselves. The pricing is more expensive if you have large volumes of data."

"The solution is a good value for batch processing and huge workloads."

"My smallest project is around a hundred euros, and my most expensive is just under a thousand euros a week. That is based on terabytes of data processed each month."

"We only pay for the Azure compute behind the solution."

More Databricks pricing and cost advice

See which vendors are best for you

Use our free recommendation engine to learn which Streaming Analytics solutions are best for your needs.

See recommendations

814,763 professionals have used our research since 2012.

Top Industries

By visitors reading reviews

Financial Services Firm

23%

Computer Software Company

20%

University

Manufacturing Company

Financial Services Firm

16%

Computer Software Company

12%

Manufacturing Company

Healthcare Company

Company Size

By reviewers

Large Enterprise

Midsize Enterprise

Small Business

Questions from the Community

What do you like most about Apache Spark Streaming?

Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows.

What needs improvement with Apache Spark Streaming?

The product's event handling capabilities, particularly compared to Kaspersky, need improvement. Integrating event-level streaming capabilities could be beneficial. This aligns with the idea of exp...

What is your primary use case for Apache Spark Streaming?

I've used it more for ETL. It's useful for creating data pipelines, streaming datasets, generating synthetic data, synchronizing data, creating data lakes, and loading and unloading data is fast an...

Which do you prefer - Databricks or Azure Machine Learning Studio?

Databricks gives you the option of working with several different languages, such as SQL, R, Scala, Apache Spark, or Python. It offers many different cluster choices and excellent integration with ...

How would you compare Databricks vs Amazon SageMaker?

We researched AWS SageMaker, but in the end, we chose Databricks. Databricks is a Unified Analytics Platform designed to accelerate innovation projects. It is based on Spark so it is very fast. It...

Which would you choose - Databricks or Azure Stream Analytics?

Databricks is an easy-to-set-up and versatile tool for data management, analysis, and business analytics. For analytics teams that have to interpret data to further the business goals of their orga...

Spring Cloud Data Flow vs Apache Spark Streaming

Comparisons

Compared 21% of the time

Amazon Kinesis vs Apache Spark Streaming

Compared 19% of the time

Apache Pulsar vs Apache Spark Streaming

Compared 16% of the time

Azure Stream Analytics vs Apache Spark Streaming

Compared 16% of the time

Apache Flink vs Apache Spark Streaming

Compared 7% of the time

More Apache Spark Streaming Competitors

Dataiku vs Databricks

Compared 11% of the time

Informatica PowerCenter vs Databricks

Compared 8% of the time

Amazon SageMaker vs Databricks

Compared 8% of the time

Dremio vs Databricks

Compared 8% of the time

Microsoft Power BI vs Databricks

Compared 6% of the time

More Databricks Competitors

Product Reports

Download Apache Spark Streaming product report

Streaming Analytics

October 2024

Download Databricks product report

November 2024

Also Known As

Spark Streaming

Databricks Unified Analytics, Databricks Unified Analytics Platform, Redash

Learn More

Apache

Overview

Spark Streaming makes it easy to build scalable fault-tolerant streaming applications.

Databricks is utilized for advanced analytics, big data processing, machine learning models, ETL operations, data engineering, streaming analytics, and integrating multiple data sources.

Organizations leverage Databricks for predictive analysis, data pipelines, data science, and unifying data architectures. It is also used for consulting projects, financial reporting, and creating APIs. Industries like insurance, retail, manufacturing, and pharmaceuticals use Databricks for data management and analytics due to its user-friendly interface, built-in machine learning libraries, support for multiple programming languages, scalability, and fast processing.

What are the key features of Databricks?

User-friendly interface: Simplifies operations and usability.
Built-in machine learning libraries: Facilitates machine learning tasks.
Support for multiple programming languages: Enhances flexibility.
Scalability: Efficiently handles growing data needs.
Fast processing: Improves performance and speed.
Automated optimization: Reduces manual efforts.
Data visualization: Provides insightful visuals.
Collaborative features: Enhances teamwork.
Delta Lake performance: Boosts data management.
Seamless cluster management: Simplifies system operations.

What are the benefits or ROI to look for in Databricks reviews?

Efficiency in handling large datasets: Ensures smooth processing.
Interactive workspace environment: Improves user collaboration.
Integration with platforms: Provides connectivity benefits.
Performance optimization: Enhances overall system performance.
Support for data governance and security: Ensures data integrity and protection.

Databricks is implemented in insurance for risk analysis and claims processing; in retail for customer analytics and inventory management; in manufacturing for predictive maintenance and supply chain optimization; and in pharmaceuticals for drug discovery and patient data analysis. Users value its scalability, machine learning support, collaboration tools, and Delta Lake performance but seek improvements in visualization, pricing, and integration with BI tools.

Sample Customers

UC Berkeley AMPLab, Amazon, Alibaba Taobao, Kenshoo, eBay Inc.

Elsevier, MyFitnessPal, Sharethrough, Automatic Labs, Celtra, Radius Intelligence, Yesware