Azure Data Factory Reviews

Azure Data Factory can be deployed on the cloud and hybrid cloud. There have been very few deployments on private clouds.

The most valuable feature of Azure Data Factory is the core features that help you through the whole Azure pipeline or value chain.

Across the whole field of use, from accepting the ingestion and real-time SaaS ingestion for which we often use other components. These areas have been instrumental across the board.

Azure Data Factory can improve the transformation features. You have to do a lot of transformation activities. This is something that is just not fully covered. Additionally, the integration could improve for other tools, such as Azure Data Catalog.

I have been using Azure Data Factory for approximately four years.

The stability of Azure Data Factory is good.

I rate the scalability of Azure Data Factory a seven out of ten.

Azure Data Factory is scalable. The solution can move up and be aligned to resources or scaled down.

We have a lot of customers using the solution, approximately 100.

The support from Azure Data Factory is very good. There are some improvements needed.

I rate the support from Azure Data Factory a four out of five.

Positive

I have previously used Informatica. When comparing Informatica to Azure Data Factory, Informatica is a bit behind.

The initial setup of Azure Data Factory is not complex if you know what you are doing. If you do not know the technology you will have a problem.

Azure Data Factory gives better value for the price than other solutions such as Informatica.

I recommend this solution to others.

I rate Azure Data Factory an eight out of ten.

I use this primarily for ETL tasks.

The most valuable feature is the ease in which you can create an ETL pipeline.

The GUI is very helpful when it comes to creating pipelines. The user interface is also very fast.

The connection to Snowflake is easy. I can store data and transform it during the ETL process before sending it to Snowflake.

Azure Data Factory is a bit complicated compared to Informatica. There are a lot of connectors that are missing and there are a lot of instances where I need to create a server and install Integration Runtime.

The support and the documentation can be improved.

There are a lot of tasks that you need to write code for.

I have been using Azure Data Factory for about six months.

I have experience with Informatica and I find it easier to use. For example, there are a lot of connectors that are directly available. Also, Informatica is able to take incremental copies, but with Azure, you have to write code to do that.

I have also worked with Matillion and Fivetran, and I feel that there are a lot of things that Azure can learn from these products. For example, with Fivetran there are very good connectors for copying data between other solutions. This is unlike Azure, where a lot of the time, I have to build my own logic.

The initial setup is complex.

I would rate this solution a seven out of ten.

We use the solution for building a few warehouses using Microsoft services.

We worked on a project for the textile industry where we needed to build a data warehouse from scratch. We provided a solution using Azure Data Factory to pull data from multiple files containing certification information, such as CSV and JSON. This data was then stored in a SQL Server-based data warehouse. We built around 30 pipelines in Azure Data Factory, one for each table, to load the data into the warehouse. The Power BI team then used this data for their analysis.

For the integration task, we used Azure Data Factory for orchestration and data flows for transformation. The data flows were beneficial, allowing us to perform multiple transformations. Additionally, we utilized web API activities to log data from third-party API tools, which greatly assisted in loading the necessary data into our warehouse.

When we initiated the cluster, it took some time to start the process. Most of our time was spent ensuring the cluster was adequately set up. We transitioned from using the auto integration runtime to a custom integration runtime, which showed some improvement.

I have been using Azure Data Factory for four years.

When running the process server, we encountered frequent connection disconnect issues. These issues often stemmed from internal problems that we couldn’t resolve then, leading to repeated disruptions.

I rate the stability as seven out of ten.

20 people are using this solution daily. I rate the scalability around eight out of ten.

Customer service supported us whenever we needed it.

Positive

We have used SQL Server.

The initial setup is easy and takes four to five hours to complete.

They have reduced the infrastructure burden by 60 percent.

Pricing is reasonable when compared with other cloud providers.

We have used the Key value pair for authentication with the adoption. I can rate it around eight out of ten.

I recommend the solution.

Overall, I rate the solution a nine out of ten.

Azure Data Factory is primarily used to orchestrate workflows and move data between various sources. It supports both ETL and ELT  processes. For instance, if you have an ERP system and want to make the data available for reporting in a data lake or data warehouse, you can use Data Factory to extract data from the ERP system as well as from other sources, like CRM systems.

Data Factory allows you to pull data from multiple systems, transform it according to your business needs, and load it into a data warehouse or data lake. It also supports complex data transformations and aggregations, enabling you to generate summary and aggregate reports from the combined data. Data Factory helps you ingest data from diverse sources, perform necessary transformations, and prepare it for reporting and analysis.

I have extensive experience building things independently, with over twenty years of experience in SQL, ETL, and data-related projects. Recently, I have been using Azure Data Factory for the past two years. It has proven to be quite effective in handling large volumes of data and performing complex calculations. It allows for the creation of intricate data workflows and processes faster. Azure Data Factory is particularly useful for enterprise-level data integration activities, where you might deal with millions of records, such as in SAP environments. For example, SAP tables can contain tens or hundreds of millions of records. Managing and maintaining the quality of this data can be challenging, but Azure Data Factory simplifies these tasks significantly.

It is a powerful tool and is considered one of the leading solutions in the market, especially for handling large volumes of data. It is popular among large enterprises.

While it has a range of connectors for various systems, such as ERP systems, the support for these connectors can be lacking. Take the SAP connector, for example. When issues arise, it can be challenging to determine whether the problem is on Microsoft's side or SAP's side. This often requires working with both teams individually, which can lead to coordination issues and delays. It would be beneficial if Azure Data Factory provided better support and troubleshooting resources for these connectors, ensuring a smoother resolution of such issues.

I have been using Azure Data Factory for two years.

I rate the solution's stability a nine out of ten.

It's pretty good. There are no issues with scalability.

The support has been good.

Positive

It is straightforward to set up. However, ensuring its security requires careful configuration, which can vary depending on the organization's requirements. While the basic setup is user-friendly and doesn’t necessarily require advanced technical skills, securing the environment involves additional steps to prevent unauthorized access and ensure that data is only accessible from permitted locations. This can be more complex depending on the specific setup and organizational needs.

Setting up the infrastructure typically takes about two to three weeks and usually requires the effort of two people.

Azure Data Factory serves several important purposes. One key reason for using it is to build an enterprise data warehouse. This is crucial for centralizing data from various sources. Another reason is to gain insights from that data. By consolidating data in a unified location, you enable data scientists and engineers to analyze it and generate valuable insights.

Customers use Azure Data Factory to bring their data together, creating opportunities to understand their data better and extract actionable insights. However, simply consolidating data is not enough; the actual value comes from how you analyze and utilize it. This involves deriving insights, creating opportunities, and understanding customers better, which can significantly benefit the organization.

Pricing is fine. It's a pay-as-you-go option.

It is in the same price range as other major providers. However, costs can vary depending on enterprise agreements and relationships.

Overall, I rate the solution a nine out of ten.

I use Azure Data Factory for architecture creation, for example, loading data from Oracle DB to Azure Synapse Analytics, creating facts and dimensions using Azure Data Pipeline, and creating Azure Synapse notebooks for data transformation. 

Another use case for Azure Data Factory is dashboard creation to help customers make informed decisions.

Compared to the on-premise SSIS, Azure Data Factory has better infrastructure. It also benefits my company because you can scale the solution up or down with different resources.

Azure Data Factory is also on a pay-as-you-go or pay-as-you-use model, which is suitable for the company because my company only pays for its usage or requirement.

The solution is also very user-friendly, and the Azure Data Factory support team responds quickly whenever my team has a loading issue.

One advantage of Azure Data Factory is that it's fast, unlike SSIS and other on-premise tools.

It's also very convenient because Azure Data Factory has multiple connectors. It has sixty connectors which you can't find in SSIS. The availability of native connectors allows you to connect to several resources to analyze data streams.

I also like that you can set up your own VM and infrastructure on Azure Data Factory without any help from the IT team because it only requires a single click.

What's missing in Azure Data Factory is an Oracle connector. If you want to connect directly to the Oracle database, you must copy and transform the data. There's no Oracle connector if you want to do transformation using data flow activity, so Azure Data Factory needs more connectors for data flow transformation.

Sending out emails after a job is completed is another area for improvement in the tool.

I've been using Azure Data Factory for three years.

Azure Data Factory is a scalable tool.

We used SSIS, but its on-premise version is slower than Azure Data Factory, and Azure Data Factory, infrastructure-wise, is better, so we went with Azure Data Factory.

The initial setup for Azure Data Factory is an eight out of ten.

Manually deploying Azure Data Factory is easy and doesn't take much time, but I'm not sure how long it takes for an automated approach to deployment.

The licensing model for Azure Data Factory is good because you won't have to overpay. Pricing-wise, the solution is a five out of ten. It was not expensive, and it was not cheap. It's in the middle.

I have experience with both Azure Data Factory and SSIS.

I'm using the latest version of Azure Data Factory.

My rating for Azure Data Factory is eight out of ten.

My company is an Azure Data Factory user.

We use Azure Data Factory for data transformation, normalization, bulk uploads, data stores, and other ETL-related tasks.

Azure Data Factory allows us to create data analytic stores in a secure manner, run machine learning on our data, and easily adapt to changing schema.

The most valuable feature of Azure Data Factory is that it has a good combination of flexibility, fine-tuning, automation, and good monitoring.

The documentation could be improved. They require more detailed error reporting, data normalization tools, easier connectivity to other services, more data services, and greater compatibility with other commonly used schemas.

I would like to see a better understanding of other common schemas, as well as a simplification of some of the more complex data normalization and standardization issues.

It would be helpful to have visibility, or better debugging, and see parts of the process as they cycle through, to get a better sense of what is and isn't working.

It's essentially just a black box. There is some monitoring that can be done, but when something goes wrong, even simple fixes are difficult to troubleshoot.

I have been working with Azure Data Factory for a couple of years.

There is only one version.

Overall, I believe the stability has been good, but there have been a couple of occasions when Microsoft's resources needed to be allocated were overburdened, and we had to wait for unacceptable amounts of time to get our slot. It has now happened twice which is not ideal.

There is no limit to scalability.

We only have a few users. One is a data scientist, and the other is a data analyst.

We use it to push up various dashboards and reports, it's a transitional product for transferring, transforming, and transitioning data.

It is extensively used, and we intend to expand our use.

You don't really get that kind of support; it's more about documentation and the community support that is available. I would rate it a three out of five compared to others.

You could call them, and pay for their consulting hours directly, but for the most part, we try to figure it out or look through documentation. 

I think their documentation is lagging because it's not as popular of a tool, there's just not a lot, or as much to fall back on.

Neutral

We had only our own tools, and we switched because you get to leverage all of the work done in a SaaS or platform as a service, or however they classify it. As a result, you get more functionality, faster, for less money.

The initial setup is straightforward.

It is a working tool. You can start using it within an hour and then make changes as needed.

We only need one person to maintain the solution; it doesn't take much to keep it running.

It's not a problem; it's a platform.

We completed the deployment ourselves.

We have seen a return on investment. I can't really share many details, but for us, this becomes something that we sell back to our clients.

You pay based on your workload. Depending on how much data you process through it, the cost could range from a few hundred dollars to tens of thousands of dollars.

Pricing is comparable, it's somewhere in the middle.

There are no additional fees to the standard licensing fee.

We looked at some other tools, such as Databricks, AmazonGlue, and MuleSoft.

We already had most of our infrastructure connected to Azure in some way. So the integration of where our data resided appeared to be simpler and safer.

I believe it would be beneficial if they could find someone experienced in some of the tools that are a part of this, such as Spark, not necessarily Data Factory specifically, but some of those other tools that will be very familiar and have a very quick time for productivity. If you're used to doing things in a different way, it may take some time because there isn't as much documentation and community support as there is for some more popular tools.

I would rate Azure Data Factory a seven out of ten.

I'm a customer. I'm using Azure Data Factory.

I find that the solution integrates well with cloud technologies, which we are using for different clouds like Snowflake and AWS. It is much more flexible in terms of transferring data from on-premise or on cloud. There is no need to create different mappings for different tables. The platform has the capability to handle metadata efficiently. So all our needs are being fulfilled with the platform we have right now.

I do not have any notes for improvement.

I have used it for almost six years.

The stability is quite good.

We wanted to move on to the cloud.

The initial setup is almost not difficult for technical people.

I used to work on Informative Support Center. Now we are using Azure Data Factory. 

I rate it eight out of ten. 

We use the solution for migration. We collect data from SAP and various other sources, including multiple ERP systems. These ERP systems encompassed different versions of SAP, Dynamics, Navision, and Oracle, presenting a considerable challenge for data integration. The objective was to consolidate all data into Azure Data Factory and Data Warehouse, establishing a structured framework for reporting and analytics. The main hurdle encountered was data ingestion, particularly with SAP data, due to its significant volume. Alternative tools such as PolyBase were utilized to expedite the process, as standard SAP APIs were insufficient for loading data into Azure Data Services. Collaboration with an Azure data engineer facilitated the exploration of alternative ingestion methods. 

The most important feature is the Data Catalog. We need to define all the data fields we test. It has technical information in the Data Catalog. The main feature is data ingestion in ADF. We also extended it to PurView because PurView is an extension of the Azure data catalog. It can scan metadata. There is a limitation in ADF when setting up the data catalog.

Integration with other tools, such as SAP, could be enhanced. It still has challenges when we talk about different types of structured and non-structured datages. Azure Data Factory has data ingestion issues. There are no delays out of the box. We needed a lot of tools to make the ingestion happen because of the data structure and size of the data.

The transformation we needed to do on data was also not so easy. It was also a long process. We had a bit more capabilities for setting up the Data Catalog, but it still didn't solve the problem from the data ingestion.

I have been using Azure Data Factory as a consultant for five years.

Sometimes, we experienced some instability, mainly on injection.

I rate the solution’s stability as seven out of ten.

I rate the solution’s scalability an eight out of ten.

The support is very good.

Positive

You need to be experienced in deploying the solution. It's not so easy for a business user. Depending on the use case, it takes around six months to get a proof of concept done.

I rate the initial setup a seven out of ten, where one is easy and ten is difficult.

The pricing is visible because you pay for what you do.

The product looks quite expensive because it charges based on the size of the data. If you're not aware, your cost can be very high. If you are experienced, you know that.

I rate the product’s pricing a seven out of ten, where one is cheap and ten is expensive.

I was mainly focusing on ingestion and cataloging. Data engineers were handling data orchestration.

The tool’s maintenance is easy.

There could be a bit more clarity in the pricing structure. It should be understandable for business users. The cost is is becoming too high because users are unaware of the pricing structure. Secondly, the tool should integrate better with other tools like ERP systems.

Overall, I rate the solution a seven out of ten.