Azure Data Factory Reviews

Our company uses the solution for data ingestion. 

It is beneficial that the solution is written with Spark as the back end. 

The solution is cloud-based and integrates well with other Azure products such as Synapse Analytics.  

There are limitations when processing more than one GD file. 

Data ingestion pipelines sometimes fail because of transient issues that have to do with the cloud network. It takes more than six hours to process or ingest 300,000 records and that is a long time. 

I have been using the solution for two years. 

The solution is new in the market and pretty stable because ADF is a little more codified than AWS. Synapse Analytics adds another tool for data. 

Stability is not quite at the level of Informatica or DataStage. 

The solution is scalable. 

For multi-tenant applications connected to multiple databases, Microsoft recommends a share box and a cell post integration run time. But a run time connecting to multiple sources has limitations and requires multiple shares connecting to your data if you are ingesting it from on-premises. 

Technical support is okay. Support is contracted or partnered with various companies but is fine as a first level. 

Most of the time, technical support has to connect with product engineers who troubleshoot issues. 

The setup is not very complex but requires intake, setting up integration services, and connecting to databases like Oracle before you push it to service.

We implemented the solution in-house. 

The pricing is pay-as-you-go or reserve instance. Of the two options, reserve instance is much cheaper. 

In the cloud, everything is service based and expensive. Users should be knowledgeable enough to maximize the solution. 

For example, it makes no sense to run integration services all day if you are not ingesting data because you pay for that usage. It is important to understand how the product works to manage it accordingly and keep costs down. 

I rate the solution a six out of ten. 

We use this solution to ingest data from one of the source systems from SAP. From the SAP HANA view, we push data to our data pond and ingest it into our data warehouse.

Azure Data Factory didn't bring a lot of good when we were also using Alteryx. Alteryx is user-friendly, while Azure Data Factory uses many resources and has issues with parallel workflows. Alteryx helps you diagnose issues quicker than Azure Data Factory because it's on the cloud and has a cold start debugger.

Azure Data Factory has to wake up whenever you are trying to do testing, and it takes about four to five minutes. It's not always online to do a quick test. For example, if we want to test an Excel file to see if the formatting is correct or why the data-flow or pipeline is failing, we need to wait four to five minutes to get the cold start debugger to run. Compared to Alteryx, Azure Data Factory could be better. Nevertheless, we are using it because we have to.

Initially, when we started using it, we didn't like it because it needed to be more mature and had data-flows, so we used the traditional pipeline. After that, Azure Data Factory introduced the concept of data-flows, and it started to become more mature and look more like Alteryx. Azure Data Factory became more user-friendly when data-flows were introduced.

They introduced the concept of Flowlets, but it has bugs. Flowlets are a reusable component that allows you to create data-flows. We can configure a Flowlet as a reusable pipeline and plug it inside different data-flows, so we don't have to rewrite our code or visual transformation.

If we make any changes in our data-flow, it reverts all our changes to the original state of the Flowlet. It does not retain changes, and we must reconfigure the Flowlets repeatedly. We had these issues three months ago so things might have changed. It works fine whenever we plug it in and configure it in our data-flow, but if we make minor changes to it, the Flowlet needs to be reconfigured again and loses the configuration.

We have used this solution for about a month and a half. It is a cloud-based tool, so there are no versions. It is all deployed on Azure Cloud.

Everything is computed inside the SQL server if we're working with pipelines, so we have to be very careful when designing our solution in Azure Data Factory. Alteryx spoiled us because we never cared how it looked in the backend because all the operations were happening on the Alteryx server. But in Azure Data Factory, they run on the capacity of our data warehouse. So Azure Data Factory cannot run your queries, and it directly sends the query to the instance in the SQL server or data warehouse. So we have to be very careful about how we perform certain operations.

We need to have knowledge of SQL and how to optimize our queries. If we are calling a stored procedure, it joins one table in Alteryx. It is pretty easy, and we just put a joint tool. Suppose we want to do it with a stored procedure in the Azure Data Factory. In that case, we have to be very careful about how we write our code. So that is a challenge for our team because we were not looking into how to optimize their SQL queries when fighting queries from Azure Data Factory to the data warehouse.

In addition, the workflows were running very slow, the performance was bad, and some queries were getting timed out because we have a threshold. So we faced many challenges and had to reeducate ourselves on SQL and query optimization.

In regards to scaling, when Azure Data Factory was introduced as your Databricks, it worked similarly to Hadoop or Spark, and it had some Spark clusters in the back end that could scale it as much as it could, and speed up the performance. So it is scalable, especially with Databricks, because a lot of data-related transformations can be performed.

On my team, there are approximately 20 people who work with Azure Data Factory.

We do not have experience with customer service and support.

It does not require any installation and is more like software as a service. You need to create an instance of Azure Data Factory in Azure and configure some of the connections to your databases. You can connect to your block storages and some authentication is necessary for Azure Data Factory.

The setup is straightforward. It doesn't take much time, and it's on cloud. It requires a few clicks, and you can quickly set it up and grant access to the developer. Then the developer can go to the link and start developing within their browser.

We have a team that takes care of the cloud infrastructure, so we raise a ticket and request infrastructure, and they just exceed it based on the naming convention with the project name.

We have an entire team that takes care of the cloud infrastructure. So we raise a ticket when we need infrastructure, which is executed based on the naming convention for the project name.

The nature of our solution is not based on ROI because we are building solutions for other functions within the same organization. In addition, due to the large size of our organization and the services we provide, the ROI is not something we consistently track. It's something discussed with the management, so I can't comment on it.

The cost is based on usage and the computing resources consumed. However, since Azure Data Factory connects with so many different functionalities that Azure provides, such as Azure functions, Logic apps and others in the Azure Data Factory pipelines, additional costs can be acquired by using other tools.

We did not evaluate other options because this solution was aligned with out current work environment. 

I rate the solution a seven out of ten. The solution is good and constantly improving, but the concept of Flowlets can be reconfigured to retain the changes we make. I advise users considering this solution to thoroughly understand what Azure Data Factory is and evaluate what's available in the market. Secondly, to assess the nature of the use cases and the kind of products they will be building before deciding to choose a solution.

My clients use Data Factory to exchange information between the on-premises environment and the cloud. Data Factory moves the data, and we use other solutions like Databricks to transform and clean up the data. My teams typically consist of three or four data engineers.

I like how you can create your own pipeline in your space and reuse those creations. You can collaborate with other people who want to use your code.

DataStage is easier to learn than Data Factory because it's more visual. Data Factory has some drag-and-drop options, but it's not as intuitive as DataStage. It would be better if they added more drag-and-drop features. You can start using DataStage without knowing the code. You don't need to learn how the code works before using the solution.

I think the communication about the ADA's would be interesting to see in the platform. How to interact with those kind of information and use it on your pipelines.

I have used Data Factory for eight months.

I have never experienced downtime with Data Factory. 

It isn't that expensive to scale Data Factory up. My client can ask for more resources on the tool, and paying more is never an issue. 

I rate Azure support seven or eight out of 10. There is room for improvement. Sometimes, you don't know where the errors originate. You have to send a ticket to Azure, and they take two or three days to respond. The issue may resolve itself by then. The problem is fixed, but you don't know how to prevent it or what to do if it happens in the future. 

The data transfer has stopped a few times for unknown reasons. We don't know if the resources are insufficient or if there is a problem with the platform. By the time we hear back from Microsoft, the issue has been resolved.

Positive

Data Factory is effortless to set up.

I rate Azure Data Factory nine out of 10. When implementing Data Factory, you should document where you are building so you can pass that information. Sometimes you build something for a specific purpose, but you can use that information for other solutions. If you have a community where you are building things, you can reuse them on the platform, so don't need to build everything from scratch.

Azure Data Factory can be deployed on the cloud and hybrid cloud. There have been very few deployments on private clouds.

The most valuable feature of Azure Data Factory is the core features that help you through the whole Azure pipeline or value chain.

Across the whole field of use, from accepting the ingestion and real-time SaaS ingestion for which we often use other components. These areas have been instrumental across the board.

Azure Data Factory can improve the transformation features. You have to do a lot of transformation activities. This is something that is just not fully covered. Additionally, the integration could improve for other tools, such as Azure Data Catalog.

I have been using Azure Data Factory for approximately four years.

The stability of Azure Data Factory is good.

I rate the scalability of Azure Data Factory a seven out of ten.

Azure Data Factory is scalable. The solution can move up and be aligned to resources or scaled down.

We have a lot of customers using the solution, approximately 100.

The support from Azure Data Factory is very good. There are some improvements needed.

I rate the support from Azure Data Factory a four out of five.

Positive

I have previously used Informatica. When comparing Informatica to Azure Data Factory, Informatica is a bit behind.

The initial setup of Azure Data Factory is not complex if you know what you are doing. If you do not know the technology you will have a problem.

Azure Data Factory gives better value for the price than other solutions such as Informatica.

I recommend this solution to others.

I rate Azure Data Factory an eight out of ten.

I primarily use Data Factory for creating pipelines on cloud in terms of integrating multiple cloud services.

Data Factory's best feature is the ease of setting up pipelines for data and cloud integrations. It also has good integration with other Azure services.

Data Factory could be improved in terms of data transformations by adding more metadata extractions.

I've been using Data Factory for five years.

Data Factory's stability has improved following some initial issues.

Data Factory's scalability is good.

The initial setup was easy as it's a SaaS offering.

Data Factory is affordable.

I would give Data Factory a rating of eight out of ten.

We use Data Factory for automating ETL processes, data management, digital transformation, and scheduled automated processes. My team has about 11 people, and at least five use Data Factory. It's mostly data engineers and analysts. 

Each data analyst and engineer manages a few projects for clients. Typically, it's one person per client, but we might have two or three people managing and building out pipelines for a larger project.  

It's cloud-based, allowing multiple users to easily access the solution from the office or remote locations. I like that we can set up the security protocols for IP addresses, like allow lists. It's a pretty user-friendly product as well. The interface and build environment where you create pipelines are easy to use. It's straightforward to manage the digital transformation pipelines we build.

Data Factory has so many features that it can be a little difficult or confusing to find some settings and configurations. I'm sure there's a way to make it a little easier to navigate.

In the main ADF web portal could, there's a section for monitoring jobs that are currently running so you can see if recent jobs have failed. There's an app for working with Azure in general where you can look at some segs in your account. It would be nice if Azure had an app that lets you access the monitoring layer of Data Factory from your phone or a tablet, so you could do a quick check-in on the status of certain jobs. That could be useful.

We've been using Azure Data Factory for about three years.

I've been happy with it overall. I don't think we've had any major issues. We've been able to do what we needed, whether connecting to different data sources or setting up different types of transformations and processes. 

It's a cloud solution, so it's inherently scalable. I don't know If we have to raise the limits on resources like clusters and processing power or if it will just automatically scale up. I can't remember offhand. 

We managed the same actions with a combination of tools. We used SFTP servers to move data from one place to another. We used scripts for loading and some other stored procedures or processes for data transformation within a database. It took two or three pieces of technology or systems to manage the same types of operation. Data Factory lets us consolidate those steps into a single pipeline. 

Setting up Azure Data Factory is pretty straightforward. We had an Azure account already, and Data Factory was just something we could add as an extra service. We had to create instances and pipelines, and it took us about two weeks to get our first pipelines scheduled and running. 

We do everything in-house.

We see a return on Data Factory if we compare the time and effort that would be necessary to perform the equivalent processes manually. 

I'm not too familiar with the cost, but I believe we're reasonably happy with what we're paying. My understanding is that the cost of Data Factory is tied to consumption. It depends on the amount of data or the number of pipelines running, and the cost varies from month to month depending on the usage. 

You'll obviously pay more if you're scheduling heavy digital transformation processes to run every hour, but I don't think there are any other hidden costs or anything extra. When you set up a new account, you have a trial period that enables you to create a test pipeline or process that's typical of your use case and then do a benchmark test to see if Data Factory can achieve the efficiency you need. You'll also get some idea of how much the process will cost to run. From there, it's straightforward to do a cost evaluation or comparison to see if it's the right fit for your company. 

We were looking for a single solution, and Data Factory was the first one that interested us. I don't think we looked at many others. We were pretty set on Azure, and Data Factory seemed to fit our needs, so we didn't make a full comparison with the alternatives.

I rate Azure Data Factory nine out of 10. It isn't perfect, but it's solid. Data Factory has improved how we deal with various aspects of Azure. It has always met our needs in terms of the transformations and jobs we want to create and schedule. 

Azure Data Factory is an integration tool, an orchestration service tool. It’s for data integration for the cloud.

The solution is okay.

Some stuff can be better, however, overall it's fine.

The performance and stability are touch and go.

The deployment should be easier.

We’d like the management of the solution to run a little more smoothly.

I’ve used the solution for three to five years.

The solution could be more stable. It’s touch and go. It’s not 100%.

For Azure Data Factory, scalability doesn't mean really too much. However, in some scenarios, you can play with it a little bit.

Azure Data Factory is not for users. Is for engineers, for developers. The end user does not interact with Azure Data Factory. There might be 20 developers on the solution currently.

I don't remember a particular scenario right now where I reached out to support. However, when you work with Azure Services, here and there, you might get into some challenges, and maybe sometimes you reach out to Microsoft. That said, I don't remember a particular scenario right now.

It’s hard to describe the installation. It’s not overly complex or extremely easy.

The point is almost true for all services. If you want to do something simple and quick, then it's just a couple of clicks, and it's there. However, in real production environments, it's not like that. You have to arrange a lot of things. You have to set up a lot of things. You have to configure a lot of things correctly in an automated way. That is totally different than just a couple of clicks. You have to put in the work. If you ask me how easy it is, yeah, it is easy. However, it can also be really, really complicated depending on the scenario.

As far as I know, there isn’t any licensing per se for this solution.

I’d rate the solution eight out of ten overall.

Azure Data Factory allows us to provide BI service. We pull the data and put it into Synapse. From there, we create our dimension fact tables that are being used for reporting. 

The most valuable feature of Azure Data Factory is the improved flexibility compared to SSIS that we previously used for our ETL transformation. I also enjoy the ease of use for the backend JSON generator, the deployment solution, and the template management. All we need to do is create ARM templates.

Microsoft is constantly upgrading its product. Changes can happen every week. Every time you open Data Factory you see something new and need to study what it is. I would like to be informed about the changes ahead of time, so we are aware of what's coming.

In future releases, I would like to see Azure Data Factory simplify how the information of logs is presented. Currently, you need to do a lot of clicks and go through steps to find out what happened. It takes too much time. The log needs to be more user-friendly.

I have been using Azure Data Factory for two years.

Scalability depends on the use case. 

As far as customer service and support with Azure Data Factory, we are not always satisfied with the response time. However, once they attend to the issue, everything is good.

Positive

All we needed to do was create ARM templates and deployment is easy.

We deployed in-house. For deployment, we use ARM templates that are a part of Azure's deployment strategy. It's not only available for Data Factory, it is built in. It links with DevOps, then the ICD integration.

Azure Data Factory is a good tool. Given that the data platform ecosystem is provided by Microsoft, you know it is good.

I would rate the solution an eight out of ten overall.

I mainly use Data Factory to load data for ETL processes or to Azure Storage and for testing purposes in our business unit.

Data Factory's most valuable feature is Copy Activity.

I've been using Data Factory for around two years.

Data Factory is stable.

We've had no problems with Data Factory's scalability.

Microsoft's technical support is responsive and quick to help.

We used consultants to implement Data Factory.

Data Factory's cost is too high.

I would advise anybody thinking of implementing Data Factory to calculate their costs at the initial stage in order to have some knowledge about future costs for the whole project. I would rate Data Factory as eight out of ten.

We created data ingestion solutions. We have built interpreters, and we have data factories that pull data from our clients. They submit data via Excel spreadsheets, and we process them into a common homogeneous format.

It has helped with some automation. Instead of individual people reviewing these files, we were able to automate the ingestion process, which saved a bunch of time. It saved hours of repeated manual work.

The data mapping and the ability to systematically derive data are nice features. It worked really well for the solution we had. It is visual, and it did the transformation as we wanted.

I couldn't quite grasp it at first because it has a Microsoft footprint on it. Some of the nomenclature around sync and other things is based on how SSRS or SSIS works, which works fine if you know these products. I didn't know them. So, some of the language and some of the settings were obtuse for me to use. It could be a little difficult if you're coming from the Java or AWS platform, but if you are coming from a Microsoft background, it would be very familiar.

For some of the data, there were some issues with data mapping. Some of the error messages were a little bit foggy. There could be more of a quick start guide or some inline examples. The documentation could be better.

There were some latency and performance issues. The processing time took slightly longer than I was hoping for. I wasn't sure if that was a licensing issue or construction of how we did the product. It wasn't super clear to me why and how those occurred. There was think time between steps. I am not sure if they can reduce the latency there. 

I have been using this solution for a year and a half.

It is very stable.

It is very scalable. It is a cloud product. It is being used by business analysts, business managers, and Azure cloud architects. We have just one developer/integrator for deployment and maintenance purposes.

We have plans to increase its usage. We'll be rolling it out for other clients.

Microsoft has these things well-documented. There were videos. I was able to find answers when I needed them. To the uninitiated, it was a little difficult, but we got there.

It was of medium complexity. Because it goes to the cloud, the duration was short. The deployment was minutes and hours.

We are a consultant and integrator. You can use our company for its implementation.

I would rate this solution a nine out of ten.