Try our new research platform with insights from 80,000+ expert users

Apache Hadoop vs Databricks comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

ROI

Sentiment score
6.5
Apache Hadoop provides cost-effective data storage and processing, though ROI varies based on analytics use and sophistication.
Sentiment score
6.6
Organizations experience mixed returns from Databricks, with benefits from cost savings and efficiency, but challenges in initial migration.
When it comes to big data processing, I prefer Databricks over other solutions.
For a lot of different tasks, including machine learning, it is a nice solution.
 

Customer Service

Sentiment score
6.4
Customer service varies by Hadoop distributor, with Hortonworks rated highly; support depends on vendor, community resources, or external vendors.
Sentiment score
7.2
Databricks customer service is generally effective with prompt responses, though some report issues mainly with third-party support channels.
It's not structured support, which is why we don't use purely open-source projects without additional structured support.
Whenever we reach out, they respond promptly.
 

Scalability Issues

Sentiment score
7.6
Apache Hadoop excels in scalability, allowing easy cluster expansion and efficient data handling, ideal for varied organizational needs.
Sentiment score
7.4
Databricks is praised for efficient scalability and cloud compatibility, allowing easy resource adjustment across diverse projects and industries.
It is a distributed file system and scales reasonably well as long as it is given sufficient resources.
The patches have sometimes caused issues leading to our jobs being paused for about six hours.
 

Stability Issues

Sentiment score
7.3
Apache Hadoop's stability, rated 8/10, improves with newer versions, though minor issues exist with memory and data processing.
Sentiment score
7.7
Databricks is stable and efficient for large data, with minor issues during updates and occasional connectivity challenges.
Continuous management in the way of upgrades and technical management is necessary to ensure that it remains effective.
They release patches that sometimes break our code.
Cluster failure is one of the biggest weaknesses I notice in our Databricks.
 

Room For Improvement

Apache Hadoop needs improved usability, integration, security, support, and performance for efficient high-volume data processing and better community resources.
Databricks users desire improved UI, enhanced data visualization, better integration, clearer error messages, robust support, and comprehensive documentation.
The problem with Apache Hadoop arose when the guys that originally set it up left the firm, and the group that later owned it didn't have enough technical resources to properly maintain it.
We prefer using a small to mid-sized cluster for many jobs to keep costs low, but this sometimes doesn't support our operations properly.
We use MLflow for managing MLOps, however, further improvement would be beneficial, especially for large language models and related tools.
If I could right-click to copy absolute paths or to read files directly into a data frame, it would standardize and simplify the process.
 

Setup Cost

Enterprise Hadoop offers cost benefits but varies with deployment type and distribution, impacting smaller organizations more heavily.
Databricks pricing depends on usage, with flexibility in licensing, and can vary in competitiveness compared to other solutions.
 

Valuable Features

Apache Hadoop excels with a scalable, cost-effective system handling diverse data types, integrating with tools, and supporting big data analytics.
Databricks provides a unified platform for data engineering, machine learning, seamless cloud integration, and robust data management capabilities.
Hadoop is a distributed file system, and it scales reasonably well provided you give it sufficient resources.
Databricks' capability to process data in parallel enhances data processing speed.
Developers can share their notebooks. Git and Azure DevOps integration on the Databricks side is also very helpful.
 

Categories and Ranking

Apache Hadoop
Average Rating
7.8
Reviews Sentiment
6.7
Number of Reviews
40
Ranking in other categories
Data Warehouse (7th)
Databricks
Average Rating
8.2
Reviews Sentiment
7.0
Number of Reviews
88
Ranking in other categories
Cloud Data Warehouse (7th), Data Science Platforms (1st), Streaming Analytics (1st)
 

Featured Reviews

Sushil Arya - PeerSpot reviewer
Provides ease of integration with the IT workflow of a business
When working with Kafka, I saw that the data came in an incremental order. The incremental data processing part is still not very effective in Apache Hadoop. If the data is already there, it can be processed very effectively, especially if the data is coming in every second. If you want to know the location of some data every second, then such data is not processed effectively in Apache Hadoop. I can say that one of the features where improvements are required revolves around the licensing cost of the tool. If the tool can build some licensing structures in a pay-per-use manner, organizations can get the look and feel of Apache Hadoop. Apache Hadoop can offer a licensing structure of the product that can be seen as similar to how AWS operates. Apache Hadoop can look into the capability of processing incremental data. The tool's setup process can be a scope of improvement. Also, it is not very simple because while doing the setup, we need to do all the server settings, including port listing and firewall configurations. If we look at other products on the market, then they can be made simpler. There are certain shortcomings when it comes to the product's technical support part, making it an area where improvements are required. The time frame for the resolution is an area that needs to be improved. The overall communication part of the technical support team also needs improvement.
ShubhamSharma7 - PeerSpot reviewer
Capability to integrate diverse coding languages in a single notebook greatly enhances workflow
Databricks offers various courses that I can use, whether it's PySpark, Scala, or R. I can leverage all these courses in a single notebook, which is beneficial for clients as they can access various tools in one place whenever needed. This is quite significant. I usually work with PySpark based on client requirements. After coding, I feed the Databricks notebooks into the ADF pipeline for updates. Databricks' capability to process data in parallel enhances data processing speed. Furthermore, I can connect our Databricks notebook directly with Power BI and other visualization tools like Qlik. Once we develop code, it allows us to transform raw data into visualizations for clients using analysis diagrams, which is very helpful.
report
Use our free recommendation engine to learn which Cloud Data Warehouse solutions are best for your needs.
842,767 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
34%
Computer Software Company
11%
University
7%
Energy/Utilities Company
6%
Financial Services Firm
17%
Computer Software Company
11%
Manufacturing Company
9%
Healthcare Company
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about Apache Hadoop?
It's primarily open source. You can handle huge data volumes and create your own views, workflows, and tables. I can also use it for real-time data streaming.
What is your experience regarding pricing and costs for Apache Hadoop?
The product is open-source, but some associated licensing fees depend on the subscription level. While it might be free for students, organizations typically need to pay for their subscriptions. Th...
What needs improvement with Apache Hadoop?
Hadoop lacks OLAP capabilities. I recommend adding a Delta Lake feature to make the data compatible with ACID properties. Also, video and audio streaming import issues could be improved to ensure p...
Which do you prefer - Databricks or Azure Machine Learning Studio?
Databricks gives you the option of working with several different languages, such as SQL, R, Scala, Apache Spark, or Python. It offers many different cluster choices and excellent integration with ...
How would you compare Databricks vs Amazon SageMaker?
We researched AWS SageMaker, but in the end, we chose Databricks. Databricks is a Unified Analytics Platform designed to accelerate innovation projects. It is based on Spark so it is very fast. It...
Which would you choose - Databricks or Azure Stream Analytics?
Databricks is an easy-to-set-up and versatile tool for data management, analysis, and business analytics. For analytics teams that have to interpret data to further the business goals of their orga...
 

Comparisons

 

Also Known As

No data available
Databricks Unified Analytics, Databricks Unified Analytics Platform, Redash
 

Overview

 

Sample Customers

Amazon, Adobe, eBay, Facebook, Google, Hulu, IBM, LinkedIn, Microsoft, Spotify, AOL, Twitter, University of Maryland, Yahoo!, Cornell University Web Lab
Elsevier, MyFitnessPal, Sharethrough, Automatic Labs, Celtra, Radius Intelligence, Yesware
Find out what your peers are saying about Apache Hadoop vs. Databricks and other solutions. Updated: March 2025.
842,767 professionals have used our research since 2012.