AWS Glue vs StreamSets comparison

Amazon Web Services (AWS) and IBM are both solutions in the Cloud Data Integration category. Additionally, 91% of Amazon Web Services (AWS) users are willing to recommend the solution, compared to 100% of IBM users who would recommend it.

AWS Glue

Read 46 AWS Glue reviews

9,512 Views
6,902 Comparison Views

91% willing to recommend

StreamSets

Read 22 StreamSets reviews

2,905 Views
1,916 Comparison Views

100% willing to recommend

AWS Glue

StreamSets

Comparison Buyer's Guide

Download the report

Executive Summary

We performed a comparison between AWS Glue and StreamSets based on real PeerSpot user reviews.

Find out in this report how the two Cloud Data Integration solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.

To learn more, read our detailed AWS Glue vs. StreamSets Report (Updated: December 2024).

Buyer's Guide

AWS Glue vs. StreamSets

December 2024

Download the complete report

Helped 824,095 peers since 2012

Categories and Ranking

AWS Glue

Average Rating

7.8

Reviews Sentiment

7.0

Number of Reviews

Ranking in other categories

Cloud Data Integration (1st)

StreamSets

Average Rating

8.4

Reviews Sentiment

7.1

Number of Reviews

Ranking in other categories

Data Integration (9th)

Featured Reviews

Ajaykumar Myana

Senior Software Developer at a computer software company with 10,001+ employees

Provides serverless mechanism, easy data transformation and automated infrastructure management

We no longer had to worry much about infrastructure management because AWS Glue is serverless, and Amazon takes care of the underlying infrastructure. This allowed us to focus on the code and application logic without concerns about scaling, CPU management, or handling fluctuations in flow. The serverless nature of Glue jobs relieved us from these infrastructure-related worries.

Read full review

Reyansh Kumar

Technical Specialist at Accenture

We no longer need to hire highly skilled data engineers to create and monitor data pipelines

The things I like about StreamSets are its * overall user interface * efficiency * product features, which are all good. Also, the scheduling within the data engineering pipeline is very much appreciated, and it has a wide range of connectors for connecting to any data sources like SQL Server, AWS, Azure, etc. We have used it with Kafka, Hadoop, and Azure Data Factory Datasets. Connecting to these systems with StreamSets is very easy. You just need to configure the data sources, the paths and their configurations, and you are ready to go. It is very efficient and very easy to use for ETL pipelines. It is a GUI-based interface in which you can easily create or design your own data pipelines with just a few clicks. As for moving data into modern analytics systems, we are using it with Microsoft Power BI, AWS, and some on-premises solutions, and it is very easy to get data from StreamSets into them. No hardcore coding or special technical expertise is required. It is also a no-code platform in which you can configure your data sources and data output for easy configuration of your data pipeline. This is a very important aspect because if a tool requires code development, we need to hire software developers to get the task done. By using StreamSets, it can be done with a few clicks.

Read full review

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros

"The facility to integrate with S3 and the possibility to use Jupyter Notebook inside the pipeline are the most valuable features."

"The solution’s most valuable feature is the ETL job."

"I appreciate AWS Glue for its cost-effectiveness."

"What I like best about AWS Glue is its real-time data backup feature. Last week, there was a production push, and what used to take almost ten days to send out around fifty-six thousand emails now takes only two hours."

"The two features I find most valuable in AWS Glue are its user interface and ease of use."

"AWS Glue is a stable and easy-to-use solution."

"The most valuable feature for me is the visual interface of AWS Glue."

"Its user interface is quite good. You just need to choose some options to create a job in AWS Glue. The code-generation feature is also useful. If you don't want to customize it and simply want to read a file and store the data in the database, it can generate the code for you."

More AWS Glue pros

"The entire user interface is very simple and the simplicity of creating pipelines is something that I like very much about it. The design experience is very smooth."

"StreamSets’ data drift resilience has reduced the time it takes us to fix data drift breakages. For example, in our previous Hadoop scenario, when we were creating the Sqoop-based processes to move data from source to destinations, we were getting the job done. That took approximately an hour to an hour and a half when we did it with Hadoop. However, with the StreamSets, since it works on a data collector-based mechanism, it completes the same process in 15 minutes of time. Therefore, it has saved us around 45 minutes per data pipeline or table that we migrate. Thus, it reduced the data transfer, including the drift part, by 45 minutes."

"StreamSets data drift feature gives us an alert upfront so we know that the data can be ingested. Whatever the schema or data type changes, it lands automatically into the data lake without any intervention from us, but then that information is crucial to fix for downstream pipelines, which process the data into models, like Tableau and Power BI models. This is actually very useful for us. We are already seeing benefits. Our pipelines used to break when there were data drift changes, then we needed to spend about a week fixing it. Right now, we are saving one to two weeks. Though, it depends on the complexity of the pipeline, we are definitely seeing a lot of time being saved."

"I really appreciate the numerous ready connectors available on both the source and target sides, the support for various media file formats, and the ease of configuring and managing pipelines centrally."

"The most valuable would be the GUI platform that I saw. I first saw it at a special session that StreamSets provided towards the end of the summer. I saw the way you set it up and how you have different processes going on with your data. The design experience seemed to be pretty straightforward to me in terms of how you drag and drop these nodes and connect them with arrows."

"The ability to have a good bifurcation rate and fewer mistakes is valuable."

"The most valuable features are the option of integration with a variety of protocols, languages, and origins."

"It's very easy to integrate. It integrates with Snowflake, AWS, Google Cloud, and Azure. It's very helpful for DevOps, DataOps, and data engineering because it provides a comprehensive solution, and it's not complicated."

More StreamSets pros

Cons

"If there's a cluster-related configuration, we have to make worker notes, which is quite a headache when processing a large amount of data."

"AWS Glue would be improved by making it easier to switch from single to multi-cloud."

"There could be an enhanced way of managing pure metadata management or data cataloging."

"Cost-wise, AWS Glue is expensive, so that's an area for improvement. The process for setting up the solution was also complex, which is another area for improvement."

"The mapping area and the use of the data catalog from Glue could be better."

"On occasion, the solution's dashboard reports that a project failed due to runtime but it actually succeeded."

"I haven't looked into Glue in terms of seeking out flaws. I've not come across missing features."

"The drawbacks associated with the product stem from the fact that, based on the data volume, it can become very costly."

More AWS Glue cons

"We've seen a couple of cases where it appears to have a memory leak or a similar problem."

"The execution engine could be improved. When I was at their session, they were using some obscure platform to run. There is a controller, which controls what happens on that, but you should be able to easily do this at any of the cloud services, such as Google Cloud. You shouldn't have any issues in terms of how to run it with their online development platform or design platform, basically their execution engine. There are issues with that."

"The monitoring visualization is not that user-friendly. It should include other features to visualize things, like how many records were streamed from a source to a destination on a particular date."

"The design experience is the bane of our existence because their documentation is not the best. Even when they update their software, they don't publish the best information on how to update and change your pipeline configuration to make it conform to current best practices. We don't pay for the added support. We use the "freeware version." The user community, as well as the documentation they provide for the standard user, are difficult, at best."

"StreamSet works great for batch processing but we are looking for something that is more real-time. We need latency in numbers below milliseconds."

"StreamSets should provide a mechanism to be able to perform data quality assessment when the data is being moved from one source to the target."

"Currently, we can only use the query to read data from SAP HANA. What we would like to see, as soon as possible, is the ability to read from multiple tables from SAP HANA. That would be a really good thing that we could use immediately. For example, if you have 100 tables in SQL Server or Oracle, then you could just point it to the schema or the 100 tables and ingestion information. However, you can't do that in SAP HANA since StreamSets currently is lacking in this. They do not have a multi-table feature for SAP HANA. Therefore, a multi-table origin for SAP HANA would be helpful."

"The logging mechanism could be improved. If I am working on a pipeline, then create a job out of it and it is running, it will generate constant logs. So, the logging mechanism could be simplified. Now, it is a bit difficult to understand and filter the logs. It takes some time."

More StreamSets cons

Pricing and Cost Advice

"It is an expensive product. I rate its pricing a nine out of ten."

"I rate the product's pricing a five on a scale of one to ten, where one is a high price, and ten is a low price."

"I rate the tool's pricing a four out of ten."

"If you are using the solution for an enterprise business, it will be expensive."

"Technical support is a paid service, and which subscription you have is dependent on that. You must pay one of them, and it ranges from $15,000 to $25,000 per year."

"AWS Glue is a high-priced solution that bills the client $150,000 to $250,000 annually."

"I rate pricing an eight out of ten."

"AWS Glue follows a pay-as-you-go model, wherein the cost of the data you use will be counted as a monthly bill."

More AWS Glue pricing and cost advice

"StreamSets Data Collector is open source. One can utilize the StreamSets Data Collector, but the Control Hub is the main repository where all the jobs are present. Everything happens in Control Hub."

"It's not so favorable for small companies."

"It's not expensive because you pay per month, and the tasks you can perform with it are huge. It's reliable and cost-effective."

"There are different versions of the product. One is the corporate license version, and the other one is the open-source or free version. I have been using the corporate license version, but they have recently launched a new open-source version so that anybody can create an account and use it. The licensing cost varies from customer to customer. I don't have a lot of input on that. It is taken care of by PMO, and they seem fine with its pricing model. It is being used enterprise-wide. They seem to have got a good deal for StreamSets."

"StreamSets is an expensive solution."

"I believe the pricing is not equitable."

"Its pricing is pretty much up to the mark. For smaller enterprises, it could be a big price to pay at the initial stage of operations, but the moment you have the Seed B or Seed C funding and you want to scale up your operations and aren't much worried about the funds, at that point in time, you would need a solution that could be scaled."

"The licensing is expensive, and there are other costs involved too. I know from using the software that you have to buy new features whenever there are new updates, which I don't really like. But initially, it was very good."

More StreamSets pricing and cost advice

See which vendors are best for you

Use our free recommendation engine to learn which Cloud Data Integration solutions are best for your needs.

See recommendations

824,095 professionals have used our research since 2012.

Top Industries

By visitors reading reviews

Financial Services Firm

22%

Computer Software Company

13%

Manufacturing Company

Insurance Company

Financial Services Firm

17%

Computer Software Company

10%

Manufacturing Company

10%

Insurance Company

Company Size

By reviewers

Large Enterprise

Midsize Enterprise

Small Business

Questions from the Community

How do you select the right cloud ETL tool?

AWS Glue and Azure Data factory for ELT best performance cloud services.

See all answers

How does Talend Open Studio compare with AWS Glue?

We reviewed AWS Glue before choosing Talend Open Studio. AWS Glue is the managed ETL (extract, transform, and load) from Amazon Web Services. AWS Glue enables AWS users to create and manage jobs in...

See all answers

What are the most common use cases for AWS Glue?

AWS Glue's main use case is for allowing users to discover, prepare, move, and integrate data from multiple sources. The product lets you use this data for analytics, application development, or ma...

See all answers

What do you like most about StreamSets?

The best thing about StreamSets is its plugins, which are very useful and work well with almost every data source. It's also easy to use, especially if you're comfortable with SQL. You can customiz...

See all answers

What needs improvement with StreamSets?

We often faced problems, especially with SAP ERP. We struggled because many columns weren't integers or primary keys, which StreamSets couldn't handle. We had to restructure our data tables, which ...

See all answers

What is your primary use case for StreamSets?

StreamSets is used for data transformation rather than ETL processes. It focuses on transforming data directly from sources without handling the extraction part of the process. The transformed data...

See all answers

Comparisons

AWS Database Migration Service vs AWS Glue

Compared 35% of the time

Informatica Intelligent Data Management Cloud (IDMC) vs AWS Glue

Compared 7% of the time

Informatica PowerCenter vs AWS Glue

Compared 6% of the time

SSIS vs AWS Glue

Compared 5% of the time

Talend Open Studio vs AWS Glue

Compared 4% of the time

More AWS Glue Competitors

Informatica PowerCenter vs StreamSets

Compared 13% of the time

IBM InfoSphere DataStage vs StreamSets

Compared 10% of the time

Fivetran vs StreamSets

Compared 10% of the time

Azure Data Factory vs StreamSets

Compared 8% of the time

Oracle GoldenGate vs StreamSets

Compared 4% of the time

More StreamSets Competitors

Product Reports

Buyer's Guide

AWS Glue

December 2024

Download AWS Glue product report

Buyer's Guide

StreamSets

November 2024

Download StreamSets product report

Learn More

Amazon Web Services (AWS)

Video not available

IBM

Overview

AWS Glue is a serverless cloud data integration tool that facilitates the discovery, preparation, movement, and integration of data from multiple sources for machine learning (ML), analytics, and application development. The solution includes additional productivity and data ops tooling for running jobs, implementing business workflows, and authoring.

AWS Glue allows users to connect to more than 70 diverse data sources and manage data in a centralized data catalog. The solution facilitates visual creation, running, and monitoring of extract, transform, and load (ETL) pipelines to load data into users' data lakes. This Amazon product seamlessly integrates with other native applications of the brand and allows users to search and query cataloged data using Amazon EMR, Amazon Athena, and Amazon Redshift Spectrum.

The solution also utilizes application programming interface (API) operations to transform users' data, create runtime logs, store job logic, and create notifications for monitoring job runs. The console of AWS Glue connects all of these services into a managed application, facilitating the monitoring and operational processes. The solution also performs provisioning and management of the resources required to run users' workloads in order to minimize manual work time for organizations.

AWS Glue Features

AWS Glue groups its features into four categories - discover, prepare, integrate, and transform. Within those groups are the following features:

Automatic schema discovery: AWS Glue crawlers connect to the organization's source or target data source through a prioritized list of classifiers to determine the schema for users' data. This feature creates metadata in companies' AWS Glue Data Catalog.
Schemas for data stream management: The AWS Glue Schema Registry enables users to validate and control the evolution of streaming data through registered Apache Avro schemas for no additional charge.
Automatic scaling based on workload: This feature dynamically scales resources up and down based on workload. The feature controls job resources, removing them depending on how much the workload can be split up.
FindMatches: This feature is for machine learning-based data deduplication and cleansing, and works by finding records that are imperfect matches of each other to remove useless data copies.
Edit, debug, and test ETL code: This feature helps users who have chosen to interactively develop their ETL code by providing development endpoints for editing, debugging, and testing the code it generates for them.
AWS Glue DataBrew: An interactive, point-and-click visual interface for specialists to clean and normalize data without the need to write any code.
AWS Glue Interactive Sessions: This feature simplifies the development of data integration jobs by enabling data engineers to interactively prepare and explore data.
AWS Glue Studio Job Notebooks: This AWS Glue feature provides serverless notebooks with minimal setup, allowing developers to start working in a timely manner.
Complex ETL pipeline building: This feature allows the product to be invoked on a schedule, on demand, or based on an event, allowing users to start multiple jobs in parallel or specify dependencies to build complex ETL pipelines.
AWS Glue Studio: This AWS Glue feature allows users to visually transform data through a drag-and-drop interface. The product automatically generates the code for ETL processes for users' data.

AWS Glue Benefits

AWS Glue offers a wide range of benefits for its users. These benefits include:

Users of other AWS products can easily onboard with AWS Glue, as it is integrated across a wide range of the company's services.
The solution is serverless, which allows for a lower total cost of ownership.
AWS Glue offers more power for users, as it automates much of the effort in building, maintaining, and running ETL jobs.
The product allows customers to easily discover and search across all their AWS datasets through AWS Glue Data Catalog.
AWS Glue does not require additional payment for managing and enforcing schemas for data streams.
The solution facilitates the authority of scalable ETL jobs for beginners and non-coding experts through a drag-and-drop interface.

Reviews from Real Users

Mustapha A., a cloud data engineer at Jems Groupe, likes AWS Glue because it is a product that is great for serverless data transformations.

Liana I., CEO at Quark Technologies SRL, describes AWS Glue as a highly scalable, reliable, and beneficial pay-as-you-go pricing model.

StreamSets is a data integration platform that enables organizations to efficiently move and process data across various systems. It offers a user-friendly interface for designing, deploying, and managing data pipelines, allowing users to easily connect to various data sources and destinations. StreamSets also provides real-time monitoring and alerting capabilities, ensuring that data is flowing smoothly and any issues are quickly addressed.

Sample Customers

bp, Cerner, Expedia, Finra, HESS, intuit, Kellog's, Philips, TIME, workday

Availity, BT Group, Humana, Deluxe, GSK, RingCentral, IBM, Shell, SamTrans, State of Ohio, TalentFulfilled, TechBridge

Buyer's Guide

AWS Glue vs. StreamSets

December 2024

Free Report: AWS Glue vs. StreamSets

Find out what your peers are saying about AWS Glue vs. StreamSets and other solutions. Updated: December 2024.

DOWNLOAD NOW

824,095 professionals have used our research since 2012.

See our AWS Glue vs. StreamSets report.

See our list of best Cloud Data Integration vendors.

We monitor all Cloud Data Integration reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.