We are using it for file ingestion. Its primary role is to ingest a file from a vendor to a database.
Team Lead at a financial services firm with 5,001-10,000 employees
It can generate the code and has a good user interface, but it lacks Java support
Pros and Cons
- "Its user interface is quite good. You just need to choose some options to create a job in AWS Glue. The code-generation feature is also useful. If you don't want to customize it and simply want to read a file and store the data in the database, it can generate the code for you."
- "Currently, it supports only two languages in the background: Python and Scala. From our customization point of view, it would be helpful if it can also support Java in the background."
What is our primary use case?
What is most valuable?
Its user interface is quite good. You just need to choose some options to create a job in AWS Glue.
The code-generation feature is also useful. If you don't want to customize it and simply want to read a file and store the data in the database, it can generate the code for you.
What needs improvement?
Currently, it supports only two languages in the background: Python and Scala. From our customization point of view, it would be helpful if it can also support Java in the background.
For how long have I used the solution?
I have been using AWS Glue for three months. We have just started using these services.
Buyer's Guide
AWS Glue
November 2024
Learn what your peers think about AWS Glue. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.
What do I think about the stability of the solution?
We have not been using AWS Glue for a long time. Till now, we haven't found any issues.
How are customer service and support?
Their technical support is good. We faced an issue with AWS Glue where we had to read a flat file. In a flat file, you only have spaces. You don't have commas or anything else. AWS Glue does not directly support flat files. You need to provide it with an expression to read the file, and that expression itself has some limitation of characters. We contacted the AWS support team. They had a call with us and first tried to understand our problem and then our use case. We gave them some sample files for our use case, and they come up with a solution for this limitation. There are some custom patterns in AWS Glue that can be used. Even though they took some time, they provided the solution. If you give a file today, they will take three to four days to get back.
How was the initial setup?
It was straightforward. A lot of documentation is available on the AWS website, which can guide you through the simple steps to set it up. Its setup was easy for me.
What's my experience with pricing, setup cost, and licensing?
It is not expensive. AWS Glue works on the serverless architecture. We get charged for the time the server is up. For our use case, we have to use it once in a day, and it is not expensive for us.
What other advice do I have?
We have just recently started to use this solution. We haven't used all features properly. It is good for the features we are using. We did not find any drawbacks or limitations so far. We are already getting whatever we want from it.
I would rate AWS Glue a seven out of ten. It needs improvements in terms of Java support and the turnaround time for our problems.
Which deployment model are you using for this solution?
Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Data Engineer | Developer at Sakshath Technologies
Data integration solution that hosts metadata before the roll out of actual data
Pros and Cons
- "The key role for Glue is that it hosts our metadata before rolling out our actual data. This is the major advantage of using this solution and our clients client have been very satisfied with it."
- "The technical support for this solution could be improved. In future, we would like to connect more services like Athena or Kinesis to help control more loads of data."
What is our primary use case?
The key role of Glue is that it hosts our metadata before rolling out our actual data. This is the major advantage of using this solution and our clients client have been very satisfied with it.
What is most valuable?
The most valuable aspect of this solution is its automation and ability to sync data from the source to the solution phase.
What needs improvement?
The technical support for this solution could be improved. In future, we would like to connect more services like Athena or Kinesis to help control more loads of data.
For how long have I used the solution?
I have been using this solution for three years.
What do I think about the stability of the solution?
This is a stable solution. We have isolated the environment using containerization so that if anything goes wrong, we have higher levels of scalability and availability. To achieve this, we have configured multiple servers for testing, UAT and development.
What do I think about the scalability of the solution?
This is a scalable solution which is supported in our organization by Docker and Kubernetes. We have 2,000 users.
How are customer service and support?
We used a vendor with an internal IT team who provided us with architecture so that we could leverage those services and reach a solution. They have 50 people in the IT team, who continuously help us and monitor the things that we are working on.
How would you rate customer service and support?
Positive
How was the initial setup?
The initial setup was straightforward and took approximately one month. For deployment, we worked in two teams. One person handled all the scripting which we are developing for automation. Two other members handled the database and servers.
What's my experience with pricing, setup cost, and licensing?
This solution is affordable and there is an option to pay for the solution based on your usage.
What other advice do I have?
I would rate this solution a seven out of ten.
Which deployment model are you using for this solution?
Hybrid Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
AWS Glue
November 2024
Learn what your peers think about AWS Glue. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.
Cloud Solution Architect at a tech services company with 1-10 employees
Cost-effective and stable
Pros and Cons
- "I appreciate AWS Glue for its cost-effectiveness."
- "In terms of improvement, the performance of AWS Glue could be faster."
What is our primary use case?
AWS Glue is a versatile tool and we mostly use it for "lift and shift" server migrations.
What is most valuable?
I appreciate AWS Glue for its cost-effectiveness. The service provides a good balance between its capabilities and the cost associated with using it.
What needs improvement?
In terms of improvement, the performance of AWS Glue could be faster.
For how long have I used the solution?
I have been using AWS Glue for five years.
What do I think about the stability of the solution?
It is a stable product.
What do I think about the scalability of the solution?
It is fairly scalable.
How are customer service and support?
The partner program support is very good.
How was the initial setup?
The initial setup is not too complex. To deploy and maintain a data platform, a general data team of around four to five skilled individuals is typically required.
What's my experience with pricing, setup cost, and licensing?
For AWS Glue, there is no separate license fee. It is part of the AWS service, and you pay for its usage as part of your overall AWS bill.
What other advice do I have?
Overall, I would rate AWS Glue as an eight out of ten.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Data Engineer at a tech services company with 201-500 employees
Great for ETL and batch processing
Pros and Cons
- "AWS Glue's most valuable features are the data catalog, including crawlers and tables, and Glue Studio, which means you don't have to use custom code."
- "If there's a cluster-related configuration, we have to make worker notes, which is quite a headache when processing a large amount of data."
What is our primary use case?
I mainly use AWS Glue for ETL purposes and batch processing of data.
What is most valuable?
AWS Glue's most valuable features are the data catalog, including crawlers and tables, and Glue Studio, which means you don't have to use custom code.
What needs improvement?
There are a couple of issues with AWS Glue. First, AWS Control randomly logs off, which disturbs coding. Second, if there's a cluster-related configuration, we have to make worker notes, which is quite a headache when processing a large amount of data. In the next release, AWS Glue should include more transformations with AWS Studio.
For how long have I used the solution?
I've been using AWS Glue for around eight months.
What do I think about the stability of the solution?
AWS Glue is stable.
How are customer service and support?
AWS' technical support responds within an hour on email.
How was the initial setup?
The initial setup was very easy, with only some minimal configuration. However, there is a drawback that once we file the name of a user, it can't be changed.
What's my experience with pricing, setup cost, and licensing?
AWS Glue is quite costly, especially for small organizations. The licensing fee is around $200 per year.
What other advice do I have?
Glue supports Spark, so if you have a team that's good with Spark, definitely go with Glue. I would rate AWS Glue as eight out of ten.
Which deployment model are you using for this solution?
Private Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
Developer-Data Engineer at Collab
Good large data processing and scalable but must overcome pipeline challenges
Pros and Cons
- "The best thing about AWS Glue is its scalability and how easy it is to process a large amount of data."
- "Setting up pipelines is challenging, especially with version control and testing requirements."
What is our primary use case?
I use AWS Glue primarily for ETL jobs. In my organization, it's just me using it as we are a small company. The IT team consists of four people, and I am the data engineering specialist.
What is most valuable?
The best thing about AWS Glue is its scalability and how easy it is to process a large amount of data. It integrates well with Redshift, S3, and AWS Glue catalog.
For processing extensive data, having a managed Spark service fulfills that role. If you're already working on AWS and you need to process a lot of data that can't be handled on a single node or server, AWS Glue will serve you well. While it's quite expensive, it's valuable for large data processing needs.
What needs improvement?
Setting up pipelines is challenging, especially with version control and testing requirements. While the initial setup is easy, it doesn't accommodate more complex development needs. You might feel hesitant about changing pipelines that are already running and processing business-critical data due to limited versioning and testing capabilities.
For how long have I used the solution?
I've been using AWS Glue since 2022, so for two years.
What do I think about the stability of the solution?
The stability of AWS Glue is fine. I haven't had any problems with it.
What do I think about the scalability of the solution?
The scalability of AWS Glue is commendable.
Which solution did I use previously and why did I switch?
Previously, in different jobs, I have worked with Databricks for ETL processes. I've also utilized Lambda functions for handling smaller data. I didn’t switch to AWS Glue, but used it in a different context.
How was the initial setup?
The initial setup of AWS Glue is easy, yet not adequate for more complex requirements. If you need to do something robust, like creating a notebook, it is straightforward.
However, when dealing with complex pipelines handling critical business data, it's hard to set up versioning and testing.
What other advice do I have?
AWS Glue receives a hesitant five out of ten from me. I recommend it if you're already on AWS and need to process large data sets. However, for smaller data volumes, I would suggest Airflow because AWS Glue can be quite expensive.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Last updated: Nov 21, 2024
Flag as inappropriateBuyer's Guide
Download our free AWS Glue Report and get advice and tips from experienced pros
sharing their opinions.
Updated: November 2024
Product Categories
Cloud Data IntegrationPopular Comparisons
Informatica Intelligent Data Management Cloud (IDMC)
MuleSoft Anypoint Platform
webMethods.io
AWS Database Migration Service
Palantir Foundry
Denodo
Matillion ETL
Fivetran
SnapLogic
Elastic Search
IBM App Connect
Zapier
IBM Cloud Pak for Integration
Talend Data integration
Jitterbit Harmony
Buyer's Guide
Download our free AWS Glue Report and get advice and tips from experienced pros
sharing their opinions.
Quick Links
Learn More: Questions:
- Which is the best choice for cloud integration: AWS Glue or Informatica Intelligent Cloud Services (IICS)?
- Is AWS Glue a difficult solution to use if you are a complete beginner?
- Is AWS Glue effective for AWS-related products only?
- Why would you choose AWS Glue over other tools?
- What are the most common use cases for AWS Glue?
- How does Talend Open Studio compare with AWS Glue?
- Does AWS Glue offer more flexibility than other ETL (Extract, Transform, Load) tools in terms of data loading?
- Oracle ICS vs ODI
- What is data lake storage?
- When evaluating Cloud Data Integration, what aspect do you think is the most important to look for?