Azure Data Factory Reviews

We are currently migrating from on-prem to the cloud, and our on-prem tables are getting data from upstream. We used ADF to build a pipeline to facilitate this migration. A team of 15-20 people currently uses ADF, and more will join once it goes live.

ADF is another ETL tool similar to Informatica that can transform data or copy it from on-prem to the cloud or vice versa. Once we have the data, we can apply various transformations to it and schedule our pipeline according to our business needs. ADF integrates with Databricks. We can call our Databricks notebooks and schedule them via ADF. 

I have used Azure Data Factory for about six months.

I rate Azure Data Factory six out of 10 for stability. ADF is stable now, but we had problems recently with indexing on an SQL database. It's slow when dealing with a huge volume of data. It depends on whether the database is configured as general purpose or hyperscale. 

I rate Azure Data Factory eight out of 10 for ease of setup. The deployment time depends on the data volume. Four million records will take longer than four thousand. Migrating our full load from on-prem to the cloud took around 16-18 hours because the volume was 17 million. 

I rate ADF six out of 10 for affordability. The cost depends on the services we use. It's usage-based. 

I rate Azure Data Factory seven out of 10. Companies that want to migrate from on-prem to the cloud have lots of options. I haven't explored them all, but Azure, GCP, and AWS are essentially all the same.

The primary use case of this solution is to extract ETLS, transform and load data, and organize database synchronization.

The most valuable feature of this solution is the data flow, which is the same SQL server in important service, integration services, which is a very robust and powerful tool to transform data.

The solution can be improved by decreasing the warmup time which currently can take up to five minutes.

I have been using the solution for two years.

The solution is extremely stable.

The solution is scalable.

Previously I used AWS Glue and SSIS.

The initial setup is straightforward.

The implementation was completed in-house and is immediate because it is a native cloud tool.

I have seen an ROI with the time saved migrating data for reports.

The solution's fees are based on a pay-per-minute use plus the amount of data required to process. The more data you process the more CPUs and time is required which increases the cost of using this solution.

I give the solution ten out of ten.

The only thing you need to deploy the solution is to click on publish.

The solution includes a feature that increases the number of processors used which makes it very powerful and adds to the scalability.

We have three people using the solution in our organization and one engineer that maintains it.

I recommend to any potential user to factor in the five-minute warm-up time that is required for each execution.

We use Azure Data Factory for data integration.

The most valuable features of the solution are its ease of use and the readily available adapters for connecting with various sources.

Some known bugs and issues with Azure Data Factory could be rectified.

I have been using Azure Data Factory for about two years.

I rate the solution an eight out of ten for stability.

Azure Data Factory is a scalable solution. A team of 16 people from the data analytics team use the solution in our organization.

I rate the solution an eight out of ten for scalability.

On a scale from one to ten, where one is difficult and ten is easy, I rate the solution's initial setup a seven out of ten.

A team of three people deployed Azure Data Factory in three to four days.

The solution's pricing is competitive.

We build data pipelines primarily for integration. Few of them are real-time data transfers, and few of them would be a batch-free file. These would direct the data from various sources to our data warehouse. Azure Data Factory helps build the data pipelines and adaptors.

The solution has built-in features and a control center for us to monitor the status of the pipelines. The solution's email notification also helps us in monitoring. We didn't face any challenges to set up the data pipelines. We know there are some controls, but governance is customized for the organization's requirements. We have our own policies.

Azure Data Factory is deployed on the cloud in our organization. I would recommend Azure Data Factory to other users.

Overall, I rate the solution a nine out of ten.

I can't go into specifics about the use case for Azure Data Factory, but it's for analytics related to an assessment.

What I like best about Azure Data Factory is that it allows you to create pipelines, specifically ETL pipelines.

I also like that Azure Data Factory has connectors and solves most of my company's problems. I can't recall a case where I couldn't use the solution for solving problems.

I'm also happy about the Azure Data Factory dashboard.

A room for improvement in Azure Data Factory is its speed. Parallelization also needs improvement. As for the rest of the features of Azure Data Factory, I'm happy.

I cannot suggest an additional feature I'd like to see in Azure Data Factory in the future because some of the features aren't available internally because the features undergo security evaluation first, and my organization controls which features would become available to users.

I've been using Azure Data Factory for the last two years.

We're happy with the stability of Azure Data Factory.

Azure Data Factory is scalable, with clusters available on demand. There isn't any issue with scaling the solution.

We have an internal support team and the Azure Data Factory support team. We raise tickets and follow up on those tickets, and on a scale of one to five, we'd rate support as four because sometimes there are delays. Otherwise, we are satisfied with Azure Data Factory support.

My company didn't set up Azure Data Factory as the Azure team did it.

We outsourced the implementation of Azure Data Factory directly to the Azure team.

I have no idea how much Azure Data Factory costs.

We're using AWS apart from Azure Data Factory. We're trying out Palantir Foundry as well. They are the leading service providers in the data analytics and ETL world.

I'm familiar with Palantir Foundry, but my company just recently got the Palantir Foundry license, so I'm still not using it, but checking it for shortcomings.

I have experience with Azure Data Factory, too.

I'm unsure of the exact version of Azure Data Factory, but I'm using the latest version or whatever's available on Azure.

I have a vague figure of users of Azure Data Factory, but it's more than one thousand to one thousand five hundred people.

I'd tell people who want to use Azure Data Factory that Microsoft offers excellent courses, ESI (Enterprise Skill Initiatives). You should register and take the courses. Azure Data Factory is a solution I'd recommend to others.

I'd rate Azure Data Factory as nine out of ten because it has a lot of connectors, even custom connectors, for data onboarding. It can also integrate with Spark notebooks and allows my organization to parallelize code. Azure Data Factory also has provisions for Spark and SQL scripts or any scripts, plus the infrastructure is highly scalable, so it's a nine for me.

My organization is a customer of Azure Data Factory.

My main use case for Azure Data Factory is to pull data from on-premises systems. Most data transformation is done through Databricks, but Data Factory mainly pulls data into different services.

The solution handles large volumes of data very well. One of its best features is its ability to integrate data end-to-end, from pulling data from the source to accessing Databricks. This makes it quite useful for our needs.

The main challenge with implementing Azure Data Factory is that it processes data in batches, not near real-time. To achieve near real-time processing, we need to schedule updates more frequently, which can be an issue. Its interface needs to be lighter. 

One specific issue is with parallel executions. When running parallel executions for multiple tables, I noticed a performance slowdown.

I have been working with the product for five years. 

We haven't faced any issues with the tool's stability. 

The solution can handle large datasets. 

I am satisfied with Microsoft's support. They provide solutions to our challenges. 

Positive

The solution is cheap. 

I rate the overall product an eight out of ten. 

We use the solution for migration. We collect data from SAP and various other sources, including multiple ERP systems. These ERP systems encompassed different versions of SAP, Dynamics, Navision, and Oracle, presenting a considerable challenge for data integration. The objective was to consolidate all data into Azure Data Factory and Data Warehouse, establishing a structured framework for reporting and analytics. The main hurdle encountered was data ingestion, particularly with SAP data, due to its significant volume. Alternative tools such as PolyBase were utilized to expedite the process, as standard SAP APIs were insufficient for loading data into Azure Data Services. Collaboration with an Azure data engineer facilitated the exploration of alternative ingestion methods. 

The most important feature is the Data Catalog. We need to define all the data fields we test. It has technical information in the Data Catalog. The main feature is data ingestion in ADF. We also extended it to PurView because PurView is an extension of the Azure data catalog. It can scan metadata. There is a limitation in ADF when setting up the data catalog.

Integration with other tools, such as SAP, could be enhanced. It still has challenges when we talk about different types of structured and non-structured datages. Azure Data Factory has data ingestion issues. There are no delays out of the box. We needed a lot of tools to make the ingestion happen because of the data structure and size of the data.

The transformation we needed to do on data was also not so easy. It was also a long process. We had a bit more capabilities for setting up the Data Catalog, but it still didn't solve the problem from the data ingestion.

I have been using Azure Data Factory as a consultant for five years.

Sometimes, we experienced some instability, mainly on injection.

I rate the solution’s stability as seven out of ten.

I rate the solution’s scalability an eight out of ten.

The support is very good.

Positive

You need to be experienced in deploying the solution. It's not so easy for a business user. Depending on the use case, it takes around six months to get a proof of concept done.

I rate the initial setup a seven out of ten, where one is easy and ten is difficult.

The pricing is visible because you pay for what you do.

The product looks quite expensive because it charges based on the size of the data. If you're not aware, your cost can be very high. If you are experienced, you know that.

I rate the product’s pricing a seven out of ten, where one is cheap and ten is expensive.

I was mainly focusing on ingestion and cataloging. Data engineers were handling data orchestration.

The tool’s maintenance is easy.

There could be a bit more clarity in the pricing structure. It should be understandable for business users. The cost is is becoming too high because users are unaware of the pricing structure. Secondly, the tool should integrate better with other tools like ERP systems.

Overall, I rate the solution a seven out of ten.

The primary use case is to connect to various different data sets and do an EAT into our data warehouse.

We have been using drivers to connect to various data sets and consume data. The solution gives everything under one roof, which is an important feature.

We require Azure Data Factory to be able to connect to Google Analytics.

I have been using the solution for two years.

The solution is stable.

The solution is scalable.

We had a few technical calls with the Microsoft technical support team for some issues that we were facing, which they helped us resolve.

Positive

The initial setup is straightforward and the team is able to deploy between six and seven days.

The implementation was completed in-house.

The cost is based on the amount of data sets that we are ingesting. The more data we ingest the more we pay.

I give the solution a nine out of ten. We have been happy with all the customer implementations, and the customers are satisfied with the ADF pipelines. We are also currently examining the Synapse pipelines, which are likely similar.

We have six developers using the solution in our organization.

People should use the solution for two reasons. Firstly, we can switch off any data pipelines we set up to save costs. Secondly, there are several connectors available in one place, including most standard connectors.

Our primary use case for the solution is data integration and we deploy it only on Azure.

When we were integrating the Ports product with our internal data warehouse, we had to update all the reports to our internal data warehouse on the Ports system database. However, they were not given access to the database company, and they dump some files or provide you with them. In one case, they were providing files. In another case, they provided some APIs where you need to call in a batch of thousands of records multiple times. It works very well with Azure Data Factory to pull the records, parse them quickly and post them in the database and data warehouse.

We have found the bulk load feature very valuable.

The only challenge with Azure Data Factory is its exception-handling mechanism. When the record fails, it's tough to identify and log.

We have been using the solution for a year and a half and are currently using the latest version.

The solution is scalable and we intend to further increase its usage in the future.

I rate customer service and support an eight out of ten.

Positive

We previously used different solutions.

The initial setup is straightforward.

The implementation was done in-house.

I cannot comment on licensing costs because I was not involved.

I rate the solution a six out of ten. The solution is good but its exception handling and logging mechanisms can be improved. I advice users considering this solution to go for it especially if their integrations are heavy on the data side.