Try our new research platform with insights from 80,000+ expert users

AWS Data Pipeline [EOL] vs AWS Glue comparison

 

Comparison Buyer's Guide

Executive Summary
 

Categories and Ranking

AWS Data Pipeline [EOL]
Average Rating
8.0
Number of Reviews
2
Ranking in other categories
No ranking in other categories
AWS Glue
Average Rating
7.8
Reviews Sentiment
7.1
Number of Reviews
45
Ranking in other categories
Cloud Data Integration (1st)
 

Featured Reviews

Geoffrey Leigh - PeerSpot reviewer
A stable, scalable, and reliable solution for moving and processing data
We're only considering enhancing the presentation layer to give a more multidimensional OLAP view that AWS seems to have decided on. Redshift with the data mart structure is like an OLAP cube. Oracle Analytics Cloud is an over-code killer and is not what we need. I was looking at Mondrian, which used to be part of the open-source stack from another vendor that works. Still, I am also looking at some of the other OLAP environments like Kaiser and perhaps decided to go to Azure with Microsoft Azure analysis cloud, but that's not multidimensional either as SSAS used to be. We tried the Mondrian, and that didn't perform how we expected. So, we are looking at resetting something to perform as an OLAP in the cloud, particularly AWS, so that we might consider an Azure solution.
Ajaykumar Myana - PeerSpot reviewer
Provides serverless mechanism, easy data transformation and automated infrastructure management
We no longer had to worry much about infrastructure management because AWS Glue is serverless, and Amazon takes care of the underlying infrastructure. This allowed us to focus on the code and application logic without concerns about scaling, CPU management, or handling fluctuations in flow. The serverless nature of Glue jobs relieved us from these infrastructure-related worries.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"It is a stable solution...It is a scalable solution."
"The most valuable feature of the solution is that orchestration and development capabilities are easier with the tool."
"The most valuable feature of AWS Glue is scalability."
"Transformations are valuable because you can modify or override complex data logic from an open source or Spark to solve issues."
"What I like best about AWS Glue is its real-time data backup feature. Last week, there was a production push, and what used to take almost ten days to send out around fifty-six thousand emails now takes only two hours."
"It is a stable and scalable solution."
"AWS Glue is very quick to start without cold starts, unlike AWS Lambda."
"AWS Glue is fast and managed by AWS. Hence, you don't have to worry about capacity and the performance of Glue jobs. It has integrations with other data stores of AWS. The product offers metadata management, logging, and ETL processing capabilities. It comes with a powerful feature, Glue Studio, which helps to do queries interactively within the community. It is a managed service and very secure. Another popular and mature service is S3."
"I like its integration and ability to handle all data-related tasks."
"Data catalog and triggers are the two best features for me. AWS Glue has its own data catalog, which makes it great and really easy to use. Triggers are also really good for scheduling the ETL process."
 

Cons

"It's almost semi-automatic because you must review and approve code push, which works well. Still, we had many problems getting there during the deployment process, but we got there."
"The user-defined functions have shortcomings in AWS Data Pipeline."
"The interface for AWS Glue could improve, they do not put a lot of details. You can write the code, in PySpark or in Scala, which is a big advantage, it is only easy to use for a developer. It will be difficult for new users to enter the cloud environment."
"It would be better if it were more user-friendly. The interesting thing we found is that it was a little strange at the beginning. The way Glue works is not very straightforward. After trying different things, for example, we used just the console to create jobs. Then we realized that things were not working as expected. After researching and learning more, we realized that even though the console creates the script for the ETL processes, you need to modify or write your own script in Spark to do everything you want it to do. For example, we are pulling data from our source database and our application database, which is in Aurora. From there, we are doing the ETL to transform the data and write the results into Redshift. But what was surprising is that it's almost like whatever you want to do, you can do it with Glue because you have the option to put together your own script. Even though there are many functionalities and many connections, you have the opportunity to write your own queries to do whatever transformations you need to do. It's a little deceiving that some options are supposed to work in a certain way when you set them up in the console, but then they are not exactly working the right way or not as expected. It would be better if they provided more examples and more documentation on options."
"AWS Glue's error handling is difficult."
"The solution could be cheaper. The price of the solution is an area that needs improvement."
"The drawbacks associated with the product stem from the fact that, based on the data volume, it can become very costly."
"The mapping area and the use of the data catalog from Glue could be better."
"Only people who can code, either in Java or Python, can use the product freely. Those who don't know Java or Python might find using AWS Glue difficult."
"The price of the solution could improve."
 

Pricing and Cost Advice

"The way we use it, I think it is fair as we're getting a good value for money compared to having a server or some other data pipeline."
"I rate the pricing between six to eight on a scale from one to ten, where one is low price, and ten is high price."
"AWS Glue uses a pay-as-you-go approach which is helpful. The price of the overall solution is low and is a great advantage."
"Its price is good. We pay as we go or based on the usage, which is a good thing for us because it is simple to forecast for the tool. It is good in terms of the financial planning of the company, and it is a good way to estimate the cost. It is also simple for our clients. In my opinion, it is one of the best tools in the market for ETL processes because of the fact that you pay as you use, which separates it from other big tools such as PowerCenter, Pentaho Data Integration, and Talend."
"I would rate the solution a six or seven on a scale of one to ten, with ten being very expensive. Specifically, I rate its pricing a six out of ten."
"AWS Glue is quite costly, especially for small organizations."
"AWS Glue is a high-priced solution that bills the client $150,000 to $250,000 annually."
"It is not expensive. AWS Glue works on the serverless architecture. We get charged for the time the server is up. For our use case, we have to use it once in a day, and it is not expensive for us."
"The pricing is a bit higher than other solutions like Athena and EC2. If the pricing becomes more scaled or flexible, it will be good because you have to pay 44 cents just for one DPU for an hour. If you increase DPUs to 5 or 10, the pricing gets multiplied. There are also some time limits like 0 to 10 minutes or 10 to 20 minutes. If the pricing is according to the minutes, it would be better because you have to limit your job to 10 minutes or 20 minutes."
"This solution is affordable and there is an option to pay for the solution based on your usage."
report
Use our free recommendation engine to learn which Cloud Data Integration solutions are best for your needs.
816,406 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
21%
Computer Software Company
20%
Government
9%
Manufacturing Company
5%
Financial Services Firm
21%
Computer Software Company
14%
Manufacturing Company
8%
Insurance Company
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
No data available
 

Questions from the Community

What do you like most about AWS Data Pipeline?
The most valuable feature of the solution is that orchestration and development capabilities are easier with the tool.
What is your experience regarding pricing and costs for AWS Data Pipeline?
I rate the pricing between six to eight on a scale from one to ten, where one is low price, and ten is high price.
What needs improvement with AWS Data Pipeline?
The user-defined functions have shortcomings in AWS Data Pipeline. The user-defined functions could be one of the areas where I can write a custom function and embed it as a part of AWS Data Pipeli...
How do you select the right cloud ETL tool?
AWS Glue and Azure Data factory for ELT best performance cloud services.
How does Talend Open Studio compare with AWS Glue?
We reviewed AWS Glue before choosing Talend Open Studio. AWS Glue is the managed ETL (extract, transform, and load) from Amazon Web Services. AWS Glue enables AWS users to create and manage jobs in...
What are the most common use cases for AWS Glue?
AWS Glue's main use case is for allowing users to discover, prepare, move, and integrate data from multiple sources. The product lets you use this data for analytics, application development, or ma...
 

Overview

 

Sample Customers

bp, Cerner, Expedia, Finra, HESS, intuit, Kellog's, Philips, TIME, workday
bp, Cerner, Expedia, Finra, HESS, intuit, Kellog's, Philips, TIME, workday
Find out what your peers are saying about Amazon Web Services (AWS), Informatica, Salesforce and others in Cloud Data Integration. Updated: November 2024.
816,406 professionals have used our research since 2012.