AWS Glue Reviews

We use AWS Glue for handling data-intensive tasks such as data lake creation, log analysis, machine learning pipelines, data warehouse population for analytics, and real-time data integration with AWS Lambda.

AWS Glue has increased efficiency and time saving. It simplifies and automates data pipeline processes, enabling faster data processing and analysis.

The AWS Glue Data Catalog provides metadata management and schema discovery. AWS Glue simplifies data transformation with automatic schema detection, incremental data updates, and integration with other AWS services.

It enables us to analyze data stored in Amazon S3 using SQL, which is manageable and cost-effective.

AWS Glue should be more reliable and faster in processing. Enhancing the speed of data processing would be beneficial.

I have been using AWS Glue for more than one year.

AWS Glue is generally considered stable and reliable for data integration, especially for larger scale production environments. Its serverless architecture and integration with other AWS services contribute to its stability.

AWS Glue is scalable because of its serverless nature, which allows for easy scaling without needing to manage any infrastructure.

The technical support from AWS is very reliable. I would rate it nine out of ten.

Positive

I did not use any other cloud data integration solutions before AWS Glue.

The initial setup process for AWS Glue involved setting up Glue resources, creating roles and permissions, and developing ETL scripts and jobs. It took about half an hour.

The deployment process required three developers.

While specific data on time and cost savings was not provided, AWS Glue's benefits include increased efficiency and time-saving.

The approximate cost for ETL jobs is about 0.44 USD, which is mostly covered by the company. Employees do not purchase AWS Glue solutions individually.

AWS Glue is highly recommended for data engineers due to its ability to build and maintain data pipelines, ensure data quality and integrity, and its integration with UI tools. It offers data preparation, machine learning integration, and governance.

I would rate it a ten out of ten.

We use AWS Glue for building ETL pipelines.

The most valuable feature of AWS Glue is that it provides a GUI format with a drag-and-drop feature. The solution provides a codeless feature or no code feature, where you can write a pipeline without adding code in AWS Glue.

If you are working on AWS services for your pipeline, AWS Glue better interacts with the AWS services than other third-party tools. The solution provides inbuilt data quality and cataloging features. AWS Glue provides a complete package, and you can do all things in one place.

AWS Glue is more costly compared to other tools like Airflow. It would be better if the solution's pricing could be reduced. The default scheduling that AWS Glue provides is not as good as Airflow. The scheduler of AWS Glue could be improved because you cannot customize it.

I have been using AWS Glue for more than three years.

AWS Glue is a stable product because it's an AWS-managed service. We can directly contact AWS for any issues we face. If there are any glitches in the version, AWS solves the issue by creating patches or version upgrades to the solution.

Around 100 to 200 people are using the solution in our organization.

As AWS Glue is a SaaS product, you don't have to set up anything. You can just create a new job, write your script, and run your job. You have to select the particular cluster or nodes you want to run and write the code. Not much admin part is required for the solution's setup.

AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution.

If you are doing a job once or twice a day, AWS Glue will not cost you much. If you run jobs for four to five times a day or hourly jobs, it will be costlier compared to other tools. If you are required to run hourly jobs five to six times a day, then using other tools would be a better option. You can choose AWS Glue if you are running jobs only one or two times a day.

Our company decided to go with AWS Glue because the tools we were using in the pipeline were AWS services only. AWS Glue easily interacts with AWS services. The jobs we were running were also not frequent.

You can use AWS Glue for learning purposes. AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution. You can learn the code by directly testing the basic spark code in any local system. Once you are comfortable that your code is working fine, then you can run your code in AWS Glue jobs. You should test the code in the local system first and then run it in AWS Glue. Testing on AWS Glue will be costly.

If a person is familiar with Spark jobs or Python jobs, they can easily learn AWS Glue. A new user will take the same amount of time to learn AWS Glue as he takes to get comfortable with Spark. Since it provides the GUI and no code thing, users can directly start using AWS Glue without having to learn any code. It's much easier to learn to use the solution.

Overall, I rate the solution a seven out of ten.

Our company has five data engineers who use the solution for metadata catalogs and ETL pipelines that are built in S3 or EC2.  

The solution integrates well with other AWS products or services. 

The solution needs to expand its 30-minute query or runtime. Sometimes it fails with certain data types such as Athena due to the limited runtime. Some large data sets run overtime during busy hours so we try to avoid failures by running data at idle times or at night. 

On occasion, the solution's dashboard reports that a project failed due to runtime but it actually succeeded. This can be quite confusing. 

I have been using the solution for one year. 

The solution is stable 99% of the time but is somewhat dependent on workloads. Heavy workloads or bigger data sets sometimes fail. Improvements are needed for stability to be consistent. 

I have not needed technical support but my colleagues have used it. I do not have relevant feedback to report. 

The setup requires some knowledge and understanding of how the solution works. 

We implemented the solution in-house. 

I rate pricing an eight out of ten. 

There are other options that offer equivalent service such as SRA and GCP but the solution is fully integrated with AWS where our data resides so we take full advantage of that. 

I rate the solution a five out of ten. 

I use AWS Glue mostly for data ingestion or data extraction from multiple sources.

AWS Glue plays a central role in AI-based solutions and machine learning workflows by efficiently handling and moving data around, contributing to streamlined operations.

One aspect that I would like to highlight is the Glue Crawler, which we utilize when working with large datasets to ensure the schema updates seamlessly without requiring end-team knowledge. 

Additionally, the pipeline orchestration and scheduling of ETL pipelines in AWS Glue are also highly effective. AWS Glue supports AI-driven projects and DAG transformations by facilitating efficient data handling required for machine learning workflows.

The point for improvement in AWS Glue would be the dynamic allocation of resources while utilizing Lambda functions. Currently, Lambda functions encounter a time-out error after fifteen minutes, necessitating an improvement in this area. Moreover, more practical examples and resources for advanced features would be beneficial for users.

With AWS Glue, I have one year of experience.

In terms of stability, there are areas that could be improved, particularly with Lambda functions and step functions. There should be better scaling for parallel computation. Despite this, AWS's overall stability is quite good.

AWS Glue receives a nine out of ten for scalability, although we feel certain functions should scale better for enhanced parallel computation.

I would rate the technical support for AWS Glue as ten out of ten. The support is high-quality.

Positive

The initial setup of AWS Glue could be straightforward for new users of AWS services, but there should be more practical examples and guides for advanced features to assist newcomers in learning complex concepts.

Currently, I work on the implementation within our development team, focusing on cloud services.

I find the pricing for AWS Glue quite affordable. For students or new users, AWS offers free credits, and as usage increases, the pay-as-you-go model provides flexibility without being expensive.

I would recommend AWS Glue to others and rate the overall solution nine out of ten.

Our primary use cases include pulling data from multiple sources and loading it into the central capacity for data transformation, integration, and processing.

I like its performance, integration, and its ability to handle all data-related tasks.

We face performance issues when using AWS Glue for data transformation and integration. It takes almost three to four hours to execute single transformations, which is a lot. We want to improve the performance to meet customer requirements.

Mainly, I am focused on improving the performance aspect because the customer is keen on this improvement.

I have been using AWS Glue for five years. I am using the latest version. 

I would rate the stability of AWS Glue a seven out of ten. There are some performance issues. 

I would rate the scalability of AWS Glue a six out of ten. There are over 25 users in my company. 

The customer service and support team is okay. 

Neutral

It is a cloud-based solution; there is no such installation procedure required. One DevOps is required for the maintenance of AWS Glue.  

Based on the customer scenario, I have previously recommended AWS Glue. Sometimes, customers directly request either Azure RapidAPI or AWS Glue. It depends on the specific business use case. Both tools have limitations, so it's hard to say which is best. If a customer already uses Microsoft products, I suggest going with Azure. As for a general rating, I would give AWS Glue a seven out of ten.

Overall, I would rate AWS Glue a seven out of ten because it's not about performance. It's because of how the tool is used. 

We are collecting some TV audience data and analyzing it.

Data catalog and triggers are the two best features for me. 

AWS Glue has its own data catalog, which makes it great and really easy to use. Triggers are also really good for scheduling the ETL process. 

The start-up time is really high right now. For instance, when you start up a new job, you have to wait for five or eight minutes before it starts. If the start-up time is reduced to one or two minutes, it will be great.

It will be better to have a direct linkage to Redshift in AWS. If we can use data catalogs from Redshift, it will be so easy to create some data catalogs. Currently, we can only use data catalogs from S3.

We have been using the AWS Glue for approximately one and a half years.

There is no problem related to stability.

Scalability is good. I can reduce or increase the number of DPUs, which I find very useful.

We are trying to increase the usage of AWS Glue because of customer needs. When the data increases, our application needs some more analyzers and user interfaces. We will increase our data analyzer and user interfaces.

I didn't take any technical support because I didn't have a big problem or issue. I just used some information from various communities and forums about the maintenance. 

The pricing is a bit higher than other solutions like Athena and EC2. If the pricing becomes more scaled or flexible, it will be good because you have to pay 44 cents just for one DPU for an hour. If you increase DPUs to 5 or 10, the pricing gets multiplied. 

There are also some time limits like 0 to 10 minutes or 10 to 20 minutes. If the pricing is according to the minutes, it would be better because you have to limit your job to 10 minutes or 20 minutes.

I would recommend AWS Glue. It is a great choice. 

I would rate this solution a nine out of ten.

I use AWS Glue primarily for ETL jobs. In my organization, it's just me using it as we are a small company. The IT team consists of four people, and I am the data engineering specialist.

The best thing about AWS Glue is its scalability and how easy it is to process a large amount of data. It integrates well with Redshift, S3, and AWS Glue catalog. 

For processing extensive data, having a managed Spark service fulfills that role. If you're already working on AWS and you need to process a lot of data that can't be handled on a single node or server, AWS Glue will serve you well. While it's quite expensive, it's valuable for large data processing needs.

Setting up pipelines is challenging, especially with version control and testing requirements. While the initial setup is easy, it doesn't accommodate more complex development needs. You might feel hesitant about changing pipelines that are already running and processing business-critical data due to limited versioning and testing capabilities.

I've been using AWS Glue since 2022, so for two years.

The stability of AWS Glue is fine. I haven't had any problems with it.

The scalability of AWS Glue is commendable.

Previously, in different jobs, I have worked with Databricks for ETL processes. I've also utilized Lambda functions for handling smaller data. I didn’t switch to AWS Glue, but used it in a different context.

The initial setup of AWS Glue is easy, yet not adequate for more complex requirements. If you need to do something robust, like creating a notebook, it is straightforward. 

However, when dealing with complex pipelines handling critical business data, it's hard to set up versioning and testing.

AWS Glue receives a hesitant five out of ten from me. I recommend it if you're already on AWS and need to process large data sets. However, for smaller data volumes, I would suggest Airflow because AWS Glue can be quite expensive.

Once you get the data and you don't know about the structure of the data, then Glue is very helpful to estimate the structure, including where is the structure, and it'll identify everything for you. It has one component that is called Glue Crawler that is quite useful for this task. It will go through segments of your data and try to guess their structure. It pops out the structure, and you can modify it according to your convenience.

It is good to basically perform the ETL when your files are stored in the S3 bucket. Glue supports other external sources also. That said, most of the time, we have basically given our proposal to clients if the data is available in S3.

The solution helps organizations gain flexibility in defining the structure of the data.

You can define and then include the original data structure and decide what the required fills are or what other ones you can omit. You can perform certain processing tasks also, and you can basically apply the multiplying factor; you can do the cleanup, et cetera, on the fly with the Glue.

The Glue Crawler can have a set of connectors, so you can utilize those connectors to connect with the external databases, which may be on-premise in different networks or maybe locally on AWS. Basically, you can use the connector to fetch the data. 

Once you have a data schema, you can start streaming or fetching the data in the particular format conversion. For example, suppose you have the text file, and you have Word in place or maybe in SQL, and you can use the connector on the fly to convert the database.

For batch processing, batch genetics, it is helpful for the ETL process.

The setup is easy. 

The solution offers a free trial. 

The solution can scale. 

It's stable.

Users only pay for what they use once they have a license. 

I haven't looked into Glue in terms of seeking out flaws. I've not come across missing features. 

I've been dealing with the solution for two or three years. I have given a lot of proposals based on customer demand.

The solution is quite stable and reliable. There are no bugs or glitches. It doesn't crash or freeze. It is reliable. 

Typically, data analytics individuals use the solution. It's not for an entire organization.

It's a scalable solution. I'd rate it ten out of ten. 

We do have plans to increase usage. We are in the process of moving many things to the cloud, and if they move onto AWS, they'll need Glue.

I've never been in touch with technical support. I can't speak to how helpful or responsive they are. 

The solution is very straightforward to set up and implement. 

I'd rate the ease of deployment at an eight or nine out of ten. However, it all depends on the circumstances. 

The deployment only takes two to three minutes. It's very fast.

Using the console, you have different sections of AWS Glue You can go and specify the input data source and output target data place. Then you need to specify the transformation. If you want to do the filtering, et cetera, you have to specify. You have the blueprint of transformation functions available also, and you can select from there and then just run it. 

I've only just explored the solution. It has not been deployed yet. 

When you are just learning and testing the solution, it is free. I cannot speak to the full cost beyond that, as I am just experimenting with the product. They do offer it to users for a limited time to try for free, however.

My understanding is you only pay for what you use, so pricing would vary based on that. You don't need to maintain a cluster and it is serverless. 

There are no extra costs beyond a standard license fee.

We are using the latest version of the solution. The solution runs on the cloud and is serverless. 

It's a good solution to use when people are not exporting analytics. If you want to perform some ETL on your data and the data is complex, then you should go for Glue. It is easy to set up.

I'd rate the solution ten out of ten.