AWS Glue Reviews

We use the solution to build tables on CSV data. We get data from some different sources, pull it in S3, and then create tables using Glue to get some metrics out of that data.

The entire use case was very easily handled or solved using AWS Glue. We had to get the files available in S3. The workflow was seamlessly integrated with the data, landing in S3, and then it detected changes made in the data. Configuring it was really easy. It gave us what we were looking for without going through a lot of hassle and that too within our budget.

The AWS Glue crawlers are valuable features. It is very versatile. It can detect the nature of the underlying data. It is quite smart, and it takes a lot of offloads rather than having to worry about configuring it or managing it.

There are output limitations and configuration of its three parts. There was a lot of trial and error that we had to go through. It is not clear how the partition discovery would have been affected by more data coming in. We've made some expensive mistakes, which, if there were any tutorials available or if there was easy documentation available with FAQs, could have been avoided. There is documentation, but it doesn't cover all.

There are three specific partition changes, and AWS Glue is tightly tied to Athena. We don't have much flexibility in managing the Athena.

AWS Glue could integrate with an AI model or a more advanced version that processes chat-based inputs rather than configuration. This would align it more closely with the functionalities of chat-based interfaces, making it easier to adopt.

I have been using AWS Glue for four to five years.

The product is stable. There are no issues, errors, or downtime. It is managed by AWS.

I rate the solution’s stability a nine out of ten.

The solution’s scalability is quite high. It is flexible and scalable. We haven't seen any challenges.

I rate the solution’s scalability an eight or nine out of ten.

The customer support is nice and very helpful without AWS premium support. There are community support, medium articles, and AWS knowledge-based articles, which provide AWS support.

Positive

I have used Spark before. We switched to AWS Glue since it is a managed solution. We were writing lines of code, managing it separately on our own, and sending the jobs on a cluster. It is taken care of by the managed service itself. We have to configure it, and then it takes care of all.

AWS Glue catalog was complex to understand. Deployment was very quick. We did some POCs. We were able to take it to production in about six to eight weeks.

We were using the console directly. There are no automated CI/CD. You can manage, create, and set up via the console using the AWS Glue service.

I rate the initial setup an eight out of ten, where one is easy, and ten is difficult.

Deployment was done in-house with the help of two data analysts.

The solution is expensive. It has a pay-as-you-go model. Whatever you are using, you are paying for that.

I rate the product’s pricing a six out of ten, where one is cheap and ten is expensive.

We are looking for Databricks. It is comparable to Spark but it was excessive for a use case. We didn't have the workload. It has a lot of additional features which we don't need. The cost is not justifiable.

Two of us are sufficient for the solution’s maintenance.

The solution is easy to set up and starts with a lot of standard data analytical use cases where we extract data. If you want something customizable, then look at other solutions because cost might be a factor for more advanced solutions.

Overall, I rate the solution a seven out of ten.

We have a lot of microservices written in Glue, which are responsible for triggering based on certain events. The solution will be responsible for another container to containerize them and run over the cloud. We use the solution for different purposes, including data computing.

You do not need many frameworks to run Glue. It's an interpreted language that does not require to be compiled at all.

It is very difficult to learn the tool and remember the syntaxes comparatively. Sometimes, I face issues integrating the solution with some third-party services or services that are not a part of Glue. Such integrations take a lot of time, and not much content is available over the internet for the same.

I have been using the solution for three to four years.

We have eight developers on our team. My team works on almost four to five Glue services. We have four team members working on Glue, including me.

I once faced an issue with Glue. There was a scenario where I wanted Glue to pick certain images, containerize them, and run over the code. That containerization integration wasn't happening successfully. I dropped a couple of messages in the community channel. I got good support from them, which helped me resolve my issue as quickly as possible. The community is very small, but the people are very helpful.

I previously worked in Java, .NET, and Python. I have extensive experience with Python and .NET. Since my organization is language-independent, we have microservices written in almost all the languages, including Glue, Python, Java, and .NET.

I'm not handling the solution's end-to-end deployment, but we have a CI/CD pipeline set up for that. The CI/CD pipeline will remain the same. It's all about how you containerize your Glue application. That is the only challenge we have faced while setting up the deployment. The rest of the configuration was pretty smooth.

Glue is not a must-have tool. You can choose Glue if you have the capability to learn Glue as quickly as possible. There are other alternatives where you will find a lot of articles, study material, and certificates over the internet apart from Glue. If you do not have any other option, go for Glue.

If Glue is not mandatory for you, go for something else because it is difficult to learn Glue and remember the syntaxes. You will need support whenever you have a bigger integration or connectivity with third-party libraries or services. You will not receive many articles or help over the internet. Although the community is available, you need to spend some time with them to make them understand the issue.

It is not easy for a beginner to learn to use the solution for the first time. There are a few videos and courses available, but it's difficult. It's not as easy as other languages in terms of content. It's hard, but you can use it once you understand the concept.

Overall, I rate the solution seven and a half out of ten.

We use AWS Glue for building ETL pipelines.

The most valuable feature of AWS Glue is that it provides a GUI format with a drag-and-drop feature. The solution provides a codeless feature or no code feature, where you can write a pipeline without adding code in AWS Glue.

If you are working on AWS services for your pipeline, AWS Glue better interacts with the AWS services than other third-party tools. The solution provides inbuilt data quality and cataloging features. AWS Glue provides a complete package, and you can do all things in one place.

AWS Glue is more costly compared to other tools like Airflow. It would be better if the solution's pricing could be reduced. The default scheduling that AWS Glue provides is not as good as Airflow. The scheduler of AWS Glue could be improved because you cannot customize it.

I have been using AWS Glue for more than three years.

AWS Glue is a stable product because it's an AWS-managed service. We can directly contact AWS for any issues we face. If there are any glitches in the version, AWS solves the issue by creating patches or version upgrades to the solution.

Around 100 to 200 people are using the solution in our organization.

As AWS Glue is a SaaS product, you don't have to set up anything. You can just create a new job, write your script, and run your job. You have to select the particular cluster or nodes you want to run and write the code. Not much admin part is required for the solution's setup.

AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution.

If you are doing a job once or twice a day, AWS Glue will not cost you much. If you run jobs for four to five times a day or hourly jobs, it will be costlier compared to other tools. If you are required to run hourly jobs five to six times a day, then using other tools would be a better option. You can choose AWS Glue if you are running jobs only one or two times a day.

Our company decided to go with AWS Glue because the tools we were using in the pipeline were AWS services only. AWS Glue easily interacts with AWS services. The jobs we were running were also not frequent.

You can use AWS Glue for learning purposes. AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution. You can learn the code by directly testing the basic spark code in any local system. Once you are comfortable that your code is working fine, then you can run your code in AWS Glue jobs. You should test the code in the local system first and then run it in AWS Glue. Testing on AWS Glue will be costly.

If a person is familiar with Spark jobs or Python jobs, they can easily learn AWS Glue. A new user will take the same amount of time to learn AWS Glue as he takes to get comfortable with Spark. Since it provides the GUI and no code thing, users can directly start using AWS Glue without having to learn any code. It's much easier to learn to use the solution.

Overall, I rate the solution a seven out of ten.

We use AWS Glue for handling data-intensive tasks such as data lake creation, log analysis, machine learning pipelines, data warehouse population for analytics, and real-time data integration with AWS Lambda.

AWS Glue has increased efficiency and time saving. It simplifies and automates data pipeline processes, enabling faster data processing and analysis.

The AWS Glue Data Catalog provides metadata management and schema discovery. AWS Glue simplifies data transformation with automatic schema detection, incremental data updates, and integration with other AWS services.

It enables us to analyze data stored in Amazon S3 using SQL, which is manageable and cost-effective.

AWS Glue should be more reliable and faster in processing. Enhancing the speed of data processing would be beneficial.

I have been using AWS Glue for more than one year.

AWS Glue is generally considered stable and reliable for data integration, especially for larger scale production environments. Its serverless architecture and integration with other AWS services contribute to its stability.

AWS Glue is scalable because of its serverless nature, which allows for easy scaling without needing to manage any infrastructure.

The technical support from AWS is very reliable. I would rate it nine out of ten.

Positive

I did not use any other cloud data integration solutions before AWS Glue.

The initial setup process for AWS Glue involved setting up Glue resources, creating roles and permissions, and developing ETL scripts and jobs. It took about half an hour.

The deployment process required three developers.

While specific data on time and cost savings was not provided, AWS Glue's benefits include increased efficiency and time-saving.

The approximate cost for ETL jobs is about 0.44 USD, which is mostly covered by the company. Employees do not purchase AWS Glue solutions individually.

AWS Glue is highly recommended for data engineers due to its ability to build and maintain data pipelines, ensure data quality and integrity, and its integration with UI tools. It offers data preparation, machine learning integration, and governance.

I would rate it a ten out of ten.

In my role as the global lead for AWS solutions and offerings, we work with various clients, including large-scale clients, to adopt and implement AWS cloud offerings. 

Our primary focus revolves around cloud lift-and-shift migration, modernization, re-platforming, rehosting, data architecture, design strategy, and implementing generative AI-specific solutions across different industries such as banking, capital insurance, energy utilities, manufacturing, automotive, semiconductor, and aerospace and defense. 

For example, we have implemented AWS Glue at several client locations, utilizing its serverless data integration capabilities during the data discovery process, enterprise transformation, cleansing, transforming, and centralizing data.

AWS Glue has significantly improved our data quality, enhancing the data by removing duplicates and providing timely and efficient insights. 

It also aids in real-time data processing, reducing effort and cost due to its serverless architecture. These features ensure we maintain the highest level of scalability, reliability, and security compliance.

AWS Glue is fully managed, providing an easy-to-use integration environment to create, run, and monitor ETL jobs. It's broadly compatible and seamlessly integrates with other AWS services like Amazon S3, Redshift, and Athena. It's flexible with data integration, manages various data formats (JSON, ORC, CSV, etc.), and is serverless, eliminating the need for infrastructure management.

Its ease of use, cost-effectiveness, and highly secure architecture are some of the most valuable features.

There could be an enhanced way of managing pure metadata management or data cataloging. 

Additionally, while it covers a wide range of integrations with AWS services, integrating with certain additional or legacy products is not seamless and can be complex. 

Increasing support for more programming languages and improving advanced analytics capabilities could also be beneficial.

We have been working with AWS Glue for almost three-plus years now.

We haven't faced any stability issues with AWS Glue. It is a scalable solution, provided that the right design principles and workload management are implemented.

AWS Glue is a scalable solution due to its serverless architecture and efficient design.

My team handles interactions with AWS for technical support, ensuring our design architectures are scalable, flexible, and well-integrated. We often reach out to the AWS team to double-check our implementation mechanisms and guidelines.

Positive

The initial setup of AWS Glue is straightforward due to its serverless architecture and fully managed nature. Specific prerequisites need to be followed, such as setting up data sources, configuring IAM permissions, creating crawlers, and running ETL jobs.

My team escalates technical questions to AWS support, ensuring our design architectures are optimal. We have a partnership with AWS, and the technical team frequently reaches out to AWS for guidance on scalability, flexibility, and integration mechanisms.

We have seen an efficient process with AWS Glue, providing the right return on investment at the right time. It ensures efficiency for our clients, giving them the desired ROI within their expected timelines.

Follow the right design principles and involve AWS at the right time to leverage the most current features and offerings from AWS Glue. Ensuring the right architecture will mitigate any issues. I'd rate the solution eight out of ten.

I had the source data, which was unstructured and non-fixable, and my responsibility was to convert it into structured data. For this task, I used PySpark as the programming language. With Python, I implemented the creation of a data frame using Glue jobs. Since Glue jobs are a serverless mechanism, I deployed my code into the Glue job, and that's how I got the job done.

We no longer had to worry much about infrastructure management because AWS Glue is serverless, and Amazon takes care of the underlying infrastructure. This allowed us to focus on the code and application logic without concerns about scaling, CPU management, or handling fluctuations in flow. The serverless nature of Glue jobs relieved us from these infrastructure-related worries.

The most valuable feature of AWS Glue for me is the ability to write PySpark code and execute it within Glue. I use Glue functions to publish parameters and interact with Glue. 

In terms of performance, if they can further optimize the execution time for serverless jobs, it would be a welcome improvement. Faster code execution would be beneficial. If AWS could enhance the serverless execution capabilities, like increasing CPU, RAM, and processing speed, that would be great.

I have been using AWS Glue for a year. 

In terms of stability, I haven't experienced any issues with AWS Glue. It has been quite reliable in my usage, and I would rate it around nine on a scale of ten.

AWS Glue is highly scalable and reliable. The serverless nature ensures that scaling is automatically managed by AWS. It's one of the strengths of Glue, and there is no doubt about its scalability.

The customer service and support were responsive and helpful in resolving the issues I had at that time. The response time was quick. So, overall, the support was okay.

Positive

Our admin team was responsible for setting up the solution. 

The pricing is a bit higher than EMR. EMR is a managed Hadoop and Spark platform. AWS Glue has a higher price. 

I found that AWS Lambda could be a possible alternative, but since AWS Lambda has a 15-minute execution limit, we opted for AWS Glue, which does not have any execution limits. I don't see any other service that can do what AWS Glue does. 

Other services that could potentially do the same job as AWS Glue would require your application to be deployed in Elastic Beanstalk, EC2, or Container Services, which is another task altogether. Since AWS Glue is already doing its job perfectly, I don't think any other service can do the same job as AWS Glue.

I would recommend that new users refer to the AWS documentation. The documentation is very well-written and easy to understand. Even new users with no prior experience with AWS should be able to get up and running quickly. I would also recommend that new users learn Python.

Overall, I would rate the solution a nine out of ten.

Our primary use cases include pulling data from multiple sources and loading it into the central capacity for data transformation, integration, and processing.

I like its performance, integration, and its ability to handle all data-related tasks.

We face performance issues when using AWS Glue for data transformation and integration. It takes almost three to four hours to execute single transformations, which is a lot. We want to improve the performance to meet customer requirements.

Mainly, I am focused on improving the performance aspect because the customer is keen on this improvement.

I have been using AWS Glue for five years. I am using the latest version. 

I would rate the stability of AWS Glue a seven out of ten. There are some performance issues. 

I would rate the scalability of AWS Glue a six out of ten. There are over 25 users in my company. 

The customer service and support team is okay. 

Neutral

It is a cloud-based solution; there is no such installation procedure required. One DevOps is required for the maintenance of AWS Glue.  

Based on the customer scenario, I have previously recommended AWS Glue. Sometimes, customers directly request either Azure RapidAPI or AWS Glue. It depends on the specific business use case. Both tools have limitations, so it's hard to say which is best. If a customer already uses Microsoft products, I suggest going with Azure. As for a general rating, I would give AWS Glue a seven out of ten.

Overall, I would rate AWS Glue a seven out of ten because it's not about performance. It's because of how the tool is used. 

We use the solution to do the usual type of transformations that before required ETL. It's mostly transformation-type purposes that we have, including transforming data from source to target. Also, we are replacing the usual ETLs with Glue, for example.

The solution has helped the organization mostly with migration. When we migrate from mostly on-premises to the cloud, Glue replaces a more expensive item that's also available in the cloud. Lots of programmers can understand Glue because they can write scripts in Python and PySpark, and there are quite a few programmers that know how to program in those languages. With the previous components that did similar things on-premise, you needed specialized knowledge.

The fact that we can use PySpark to program them is great. 

It's fairly straightforward as a product; it's not very complicated. 

The fact that Amazon offers it and it's a quick way of getting things done, and the fact that it doesn't require a lot of training is very positive attributes.

The initial setup is straightforward.  

It can scale.

The mapping area and the use of the data catalog from Glue could be better. I would say those two are the main things we'd like to see improvements on.

The solution needs support for big data.

As I understand it, Glue is based on Lambdas and Lambdas have some limitations as far as running them continuously. Sometimes they get dropped, and they have to be reinitialized.

I've been using the solution for about two years. 

So far, the solution has been stable. For everything that we've done, we didn't run into any problems, so it was fairly stable for us.

The solution can scale. I don't know exactly at what scale. I was using it for a medium-scale type solution. We never tested it with extremely large scales, so I don't know how expansive it can be. That said, since it's in the cloud, if you need more scale, it would be easy enough to add additional Glue components. At least, in theory, it should be fairly scalable.

Indirectly, we have hundreds of people on the solution. 

At this time, I'm not sure if any clients plan to increase usage. 

Since the solution is so easy to use, we have not needed the help of technical support. 

We've previously used many different solutions, including Informatica, Data Storage from IBM, and SSIS.

The solution is easy to set up. It's not overly complex.

At a minimum, a company would need one to two people to handle the deployment. If it is a larger scale of deployment, they might need more personnel. 

As a consultant, I assist with the implementation. 

Likely, a company would receive an ROI as it's cheaper than the alternatives.

I was not involved in the cost negotiation process. 

Our clients typically have chosen Glue from the start. I was not involved in the evaluation process. 

We are using one of the latest versions of the solution. It's about two years old. 

Depending on the number of data sources, the variety of data sources, and the variety of targets they will have, I might recommend the solution. What they have and plan to do will dictate whether Glue is a good solution or whether they would require something more sophisticated - such as Databricks. For example, if you have big data, then Databricks is probably a better solution to do ETL.

I'd rate the solution seven out of ten.