Try our new research platform with insights from 80,000+ expert users

AWS Lambda vs Apache Spark comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Categories and Ranking

Apache Spark
Ranking in Compute Service
4th
Average Rating
8.4
Reviews Sentiment
7.7
Number of Reviews
66
Ranking in other categories
Hadoop (1st), Java Frameworks (2nd)
AWS Lambda
Ranking in Compute Service
1st
Average Rating
8.4
Reviews Sentiment
7.5
Number of Reviews
84
Ranking in other categories
No ranking in other categories
 

Mindshare comparison

As of April 2025, in the Compute Service category, the mindshare of Apache Spark is 11.2%, up from 9.7% compared to the previous year. The mindshare of AWS Lambda is 21.0%, down from 23.2% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Compute Service
 

Featured Reviews

Ilya Afanasyev - PeerSpot reviewer
Reliable, able to expand, and handle large amounts of data well
We use batch processing. It works well with our formats and file versions. There's a lot of functionality. In our pipeline each hour, we make a copy of data from MongoDB, of the changes from MongoDB to some specific file. Each time pipeline copied all of the data, it would do it each time without changes to all of the tables. Tables have a lot of data, and in the last MongoDB version, there is a possibility to read only changed data. This reduced the cost and configuration of the cluster, and we saved about $150,000. The solution is scalable. It's a stable product.
Wai L Lin O - PeerSpot reviewer
A serverless solution with easy integration features
We use AWS Lambda because it provides a solution for our needs without requiring us to manage our infrastructure. With the tool, we only pay for the resources we use. Additionally, it is straightforward to implement and integrates with other services like API Gateway. The tool's serverless nature has had the most significant impact on our workflow. I find it particularly attractive because it eliminates the need for managing servers. In my previous experience, managing upgrades and updates was quite challenging. The solution's integration process with other AWS services was relatively easy. We primarily use AWS services such as EventBridge for scheduling processes and log management.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"It provides a scalable machine learning library."
"We use Spark to process data from different data sources."
"I like that it can handle multiple tasks parallelly. I also like the automation feature. JavaScript also helps with the parallel streaming of the library."
"The most significant advantage of Spark 3.0 is its support for DataFrame UDF Pandas UDF features."
"Apache Spark is known for its ease of use. Compared to other available data processing frameworks, it is user-friendly."
"Now, when we're tackling sentiment analysis using NLP technologies, we deal with unstructured data—customer chats, feedback on promotions or demos, and even media like images, audio, and video files. For processing such data, we rely on PySpark. Beneath the surface, Spark functions as a compute engine with in-memory processing capabilities, enhancing performance through features like broadcasting and caching. It's become a crucial tool, widely adopted by 90% of companies for a decade or more."
"I like Apache Spark's flexibility the most. Before, we had one server that would choke up. With the solution, we can easily add more nodes when needed. The machine learning models are also really helpful. We use them to predict energy theft and find infrastructure problems."
"The features we find most valuable are the machine learning, data learning, and Spark Analytics."
"AWS Lambda is entirely stable."
"AWS Lambda has improved our productivity and functionality."
"Some of the most valuable features are that it's easy to install and use. The performance is also good."
"The initial setup is pretty easy."
"AWS Lambda is cost-effective, providing noticeable cost savings."
"What I like best about AWS Lambda is that it's feature-rich, and I appreciate that. I also like that it's stable and supports many languages."
"I can use the solution to configure and set up all the requirements for testing the application and test code."
"By using Lambda, we can use Python code and the Boto3 solution."
 

Cons

"One limitation is that not all machine learning libraries and models support it."
"The logging for the observability platform could be better."
"When using Spark, users may need to write their own parallelization logic, which requires additional effort and expertise."
"This solution currently cannot support or distribute neural network related models, or deep learning related algorithms. We would like this functionality to be developed."
"It requires overcoming a significant learning curve due to its robust and feature-rich nature."
"We use big data manager but we cannot use it as conditional data so whenever we're trying to fetch the data, it takes a bit of time."
"We've had problems using a Python process to try to access something in a large volume of data. It crashes if somebody gives me the wrong code because it cannot handle a large volume of data."
"There were some problems related to the product's compatibility with a few Python libraries."
"I wish to see better execution time in the next release."
"AWS Lambda should support additional languages."
"Lambda would benefit from a debugging feature as well."
"The feature to attach external storage, such as an S3 or elastic storage, must be added to AWS Lambda. This is its area for improvement."
"I would like to see some better integration with other providers, like Cohesity, Druva, and others. I also think the Lambda interface could be better."
"If you're running a new application with a significant load, you need to be prepared for potential bottlenecks."
"AWS Lambda is a bit difficult to set up if someone doesn't know how to code."
"AWS Lambda has a limitation where the execution time is capped at 15 minutes per task. Increasing this time would allow for handling heavier tasks more efficiently."
 

Pricing and Cost Advice

"Considering the product version used in my company, I feel that the tool is not costly since the product is available for free."
"Apache Spark is not too cheap. You have to pay for hardware and Cloudera licenses. Of course, there is a solution with open source without Cloudera."
"It is an open-source platform. We do not pay for its subscription."
"We are using the free version of the solution."
"Spark is an open-source solution, so there are no licensing costs."
"I did not pay anything when using the tool on cloud services, but I had to pay on the compute side. The tool is not expensive compared with the benefits it offers. I rate the price as an eight out of ten."
"Apache Spark is an open-source tool."
"It is quite expensive. In fact, it accounts for almost 50% of the cost of our entire project."
"AWS Lambda is cost-effective, with a minimal maintenance cost."
"The cost is based on runtime."
"AWS Lambda is a cheap solution."
"You're not paying for a server if you're not using it, which is another reason I like it. So, you're not paying if you're not using it. It scales, and you're charged based on usage. It all depends on the use case. Some can be extremely inexpensive if you have very low volume transaction rates. That way, you don't have to fire up and absorb the cost of the servers just sitting there waiting for a transaction to come through. You're only paying when you use it. So, depending upon the use model, Lambda could be highly efficient relative to an EC2 solution. You don't have to have things reallocated."
"It computes by the cycle, and it's very cheap."
"Lambda is an affordable solution. They offer free requests every month and charge per the compute time. If you are working in a big organization, usually AWS offer a savings plan where you get approximately 70% discount on pricing."
"Price-wise, AWS Lambda is very cheap. It's not free, but it's not that expensive."
"The solution's price is average."
report
Use our free recommendation engine to learn which Compute Service solutions are best for your needs.
849,190 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
27%
Computer Software Company
13%
Manufacturing Company
8%
Comms Service Provider
6%
Educational Organization
68%
Financial Services Firm
8%
Computer Software Company
5%
Manufacturing Company
3%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about Apache Spark?
We use Spark to process data from different data sources.
What is your experience regarding pricing and costs for Apache Spark?
Compared to other solutions like Doc DB, Spark is more costly due to the need for extensive infrastructure. It requires significant investment in infrastructure, which can be expensive. While cloud...
What needs improvement with Apache Spark?
The Spark solution could improve in scheduling tasks and managing dependencies. Spark alone cannot handle sequential tasks, requiring environments like Airflow scheduler or scripts. For instance, o...
Which is better, AWS Lambda or Batch?
AWS Lambda is a serverless solution. It doesn’t require any infrastructure, which allows for cost savings. There is no setup process to deal with, as the entire solution is in the cloud. If you use...
What do you like most about AWS Lambda?
The tool scales automatically based on the number of incoming requests.
What is your experience regarding pricing and costs for AWS Lambda?
AWS Lambda is cheaper compared to running an instance continuously. You only pay for what you use, making it cost-effective.
 

Comparisons

 

Overview

 

Sample Customers

NASA JPL, UC Berkeley AMPLab, Amazon, eBay, Yahoo!, UC Santa Cruz, TripAdvisor, Taboola, Agile Lab, Art.com, Baidu, Alibaba Taobao, EURECOM, Hitachi Solutions
Netflix
Find out what your peers are saying about AWS Lambda vs. Apache Spark and other solutions. Updated: April 2025.
849,190 professionals have used our research since 2012.