Azure Data Factory Reviews

We use Azure Data Factory for data integration.

The most valuable features of the solution are its ease of use and the readily available adapters for connecting with various sources.

Some known bugs and issues with Azure Data Factory could be rectified.

I have been using Azure Data Factory for about two years.

I rate the solution an eight out of ten for stability.

Azure Data Factory is a scalable solution. A team of 16 people from the data analytics team use the solution in our organization.

I rate the solution an eight out of ten for scalability.

On a scale from one to ten, where one is difficult and ten is easy, I rate the solution's initial setup a seven out of ten.

A team of three people deployed Azure Data Factory in three to four days.

The solution's pricing is competitive.

We build data pipelines primarily for integration. Few of them are real-time data transfers, and few of them would be a batch-free file. These would direct the data from various sources to our data warehouse. Azure Data Factory helps build the data pipelines and adaptors.

The solution has built-in features and a control center for us to monitor the status of the pipelines. The solution's email notification also helps us in monitoring. We didn't face any challenges to set up the data pipelines. We know there are some controls, but governance is customized for the organization's requirements. We have our own policies.

Azure Data Factory is deployed on the cloud in our organization. I would recommend Azure Data Factory to other users.

Overall, I rate the solution a nine out of ten.

We are currently migrating from on-prem to the cloud, and our on-prem tables are getting data from upstream. We used ADF to build a pipeline to facilitate this migration. A team of 15-20 people currently uses ADF, and more will join once it goes live.

ADF is another ETL tool similar to Informatica that can transform data or copy it from on-prem to the cloud or vice versa. Once we have the data, we can apply various transformations to it and schedule our pipeline according to our business needs. ADF integrates with Databricks. We can call our Databricks notebooks and schedule them via ADF. 

I have used Azure Data Factory for about six months.

I rate Azure Data Factory six out of 10 for stability. ADF is stable now, but we had problems recently with indexing on an SQL database. It's slow when dealing with a huge volume of data. It depends on whether the database is configured as general purpose or hyperscale. 

I rate Azure Data Factory eight out of 10 for ease of setup. The deployment time depends on the data volume. Four million records will take longer than four thousand. Migrating our full load from on-prem to the cloud took around 16-18 hours because the volume was 17 million. 

I rate ADF six out of 10 for affordability. The cost depends on the services we use. It's usage-based. 

I rate Azure Data Factory seven out of 10. Companies that want to migrate from on-prem to the cloud have lots of options. I haven't explored them all, but Azure, GCP, and AWS are essentially all the same.

We use this solution to ingest data from one of the source systems from SAP. From the SAP HANA view, we push data to our data pond and ingest it into our data warehouse.

Azure Data Factory didn't bring a lot of good when we were also using Alteryx. Alteryx is user-friendly, while Azure Data Factory uses many resources and has issues with parallel workflows. Alteryx helps you diagnose issues quicker than Azure Data Factory because it's on the cloud and has a cold start debugger.

Azure Data Factory has to wake up whenever you are trying to do testing, and it takes about four to five minutes. It's not always online to do a quick test. For example, if we want to test an Excel file to see if the formatting is correct or why the data-flow or pipeline is failing, we need to wait four to five minutes to get the cold start debugger to run. Compared to Alteryx, Azure Data Factory could be better. Nevertheless, we are using it because we have to.

Initially, when we started using it, we didn't like it because it needed to be more mature and had data-flows, so we used the traditional pipeline. After that, Azure Data Factory introduced the concept of data-flows, and it started to become more mature and look more like Alteryx. Azure Data Factory became more user-friendly when data-flows were introduced.

They introduced the concept of Flowlets, but it has bugs. Flowlets are a reusable component that allows you to create data-flows. We can configure a Flowlet as a reusable pipeline and plug it inside different data-flows, so we don't have to rewrite our code or visual transformation.

If we make any changes in our data-flow, it reverts all our changes to the original state of the Flowlet. It does not retain changes, and we must reconfigure the Flowlets repeatedly. We had these issues three months ago so things might have changed. It works fine whenever we plug it in and configure it in our data-flow, but if we make minor changes to it, the Flowlet needs to be reconfigured again and loses the configuration.

We have used this solution for about a month and a half. It is a cloud-based tool, so there are no versions. It is all deployed on Azure Cloud.

Everything is computed inside the SQL server if we're working with pipelines, so we have to be very careful when designing our solution in Azure Data Factory. Alteryx spoiled us because we never cared how it looked in the backend because all the operations were happening on the Alteryx server. But in Azure Data Factory, they run on the capacity of our data warehouse. So Azure Data Factory cannot run your queries, and it directly sends the query to the instance in the SQL server or data warehouse. So we have to be very careful about how we perform certain operations.

We need to have knowledge of SQL and how to optimize our queries. If we are calling a stored procedure, it joins one table in Alteryx. It is pretty easy, and we just put a joint tool. Suppose we want to do it with a stored procedure in the Azure Data Factory. In that case, we have to be very careful about how we write our code. So that is a challenge for our team because we were not looking into how to optimize their SQL queries when fighting queries from Azure Data Factory to the data warehouse.

In addition, the workflows were running very slow, the performance was bad, and some queries were getting timed out because we have a threshold. So we faced many challenges and had to reeducate ourselves on SQL and query optimization.

In regards to scaling, when Azure Data Factory was introduced as your Databricks, it worked similarly to Hadoop or Spark, and it had some Spark clusters in the back end that could scale it as much as it could, and speed up the performance. So it is scalable, especially with Databricks, because a lot of data-related transformations can be performed.

On my team, there are approximately 20 people who work with Azure Data Factory.

We do not have experience with customer service and support.

It does not require any installation and is more like software as a service. You need to create an instance of Azure Data Factory in Azure and configure some of the connections to your databases. You can connect to your block storages and some authentication is necessary for Azure Data Factory.

The setup is straightforward. It doesn't take much time, and it's on cloud. It requires a few clicks, and you can quickly set it up and grant access to the developer. Then the developer can go to the link and start developing within their browser.

We have a team that takes care of the cloud infrastructure, so we raise a ticket and request infrastructure, and they just exceed it based on the naming convention with the project name.

We have an entire team that takes care of the cloud infrastructure. So we raise a ticket when we need infrastructure, which is executed based on the naming convention for the project name.

The nature of our solution is not based on ROI because we are building solutions for other functions within the same organization. In addition, due to the large size of our organization and the services we provide, the ROI is not something we consistently track. It's something discussed with the management, so I can't comment on it.

The cost is based on usage and the computing resources consumed. However, since Azure Data Factory connects with so many different functionalities that Azure provides, such as Azure functions, Logic apps and others in the Azure Data Factory pipelines, additional costs can be acquired by using other tools.

We did not evaluate other options because this solution was aligned with out current work environment. 

I rate the solution a seven out of ten. The solution is good and constantly improving, but the concept of Flowlets can be reconfigured to retain the changes we make. I advise users considering this solution to thoroughly understand what Azure Data Factory is and evaluate what's available in the market. Secondly, to assess the nature of the use cases and the kind of products they will be building before deciding to choose a solution.

My main use case for Azure Data Factory is to pull data from on-premises systems. Most data transformation is done through Databricks, but Data Factory mainly pulls data into different services.

The solution handles large volumes of data very well. One of its best features is its ability to integrate data end-to-end, from pulling data from the source to accessing Databricks. This makes it quite useful for our needs.

The main challenge with implementing Azure Data Factory is that it processes data in batches, not near real-time. To achieve near real-time processing, we need to schedule updates more frequently, which can be an issue. Its interface needs to be lighter. 

One specific issue is with parallel executions. When running parallel executions for multiple tables, I noticed a performance slowdown.

I have been working with the product for five years. 

We haven't faced any issues with the tool's stability. 

The solution can handle large datasets. 

I am satisfied with Microsoft's support. They provide solutions to our challenges. 

Positive

The solution is cheap. 

I rate the overall product an eight out of ten. 

I use Azure Data Factory for architecture creation, for example, loading data from Oracle DB to Azure Synapse Analytics, creating facts and dimensions using Azure Data Pipeline, and creating Azure Synapse notebooks for data transformation. 

Another use case for Azure Data Factory is dashboard creation to help customers make informed decisions.

Compared to the on-premise SSIS, Azure Data Factory has better infrastructure. It also benefits my company because you can scale the solution up or down with different resources.

Azure Data Factory is also on a pay-as-you-go or pay-as-you-use model, which is suitable for the company because my company only pays for its usage or requirement.

The solution is also very user-friendly, and the Azure Data Factory support team responds quickly whenever my team has a loading issue.

One advantage of Azure Data Factory is that it's fast, unlike SSIS and other on-premise tools.

It's also very convenient because Azure Data Factory has multiple connectors. It has sixty connectors which you can't find in SSIS. The availability of native connectors allows you to connect to several resources to analyze data streams.

I also like that you can set up your own VM and infrastructure on Azure Data Factory without any help from the IT team because it only requires a single click.

What's missing in Azure Data Factory is an Oracle connector. If you want to connect directly to the Oracle database, you must copy and transform the data. There's no Oracle connector if you want to do transformation using data flow activity, so Azure Data Factory needs more connectors for data flow transformation.

Sending out emails after a job is completed is another area for improvement in the tool.

I've been using Azure Data Factory for three years.

Azure Data Factory is a scalable tool.

We used SSIS, but its on-premise version is slower than Azure Data Factory, and Azure Data Factory, infrastructure-wise, is better, so we went with Azure Data Factory.

The initial setup for Azure Data Factory is an eight out of ten.

Manually deploying Azure Data Factory is easy and doesn't take much time, but I'm not sure how long it takes for an automated approach to deployment.

The licensing model for Azure Data Factory is good because you won't have to overpay. Pricing-wise, the solution is a five out of ten. It was not expensive, and it was not cheap. It's in the middle.

I have experience with both Azure Data Factory and SSIS.

I'm using the latest version of Azure Data Factory.

My rating for Azure Data Factory is eight out of ten.

My company is an Azure Data Factory user.

My primary use case of Azure Data Factory is supporting the data migration for advanced analytics projects. 

One of the most valuable features of Azure Data Factory is the drag-and-drop interface. This helps with workflow management because we can just drag any tables or data sources we need. Because of how easy it is to drag and drop, we can deliver things very quickly. It's more customizable through visual effect. 

Data Factory could be improved by eliminating the need for a physical data area. We have to extract data using Data Factory, then create a staging database for it with Azure SQL, which is very, very expensive. Another improvement would be lowering the licensing cost. 

I have been using this solution for the past year. 

This solution is stable. We are using an Azure subscription, so there is no maintenance or direct updates, it's just always the latest version.

This solution is automatically scalable, since it's in the cloud. At my company, there were more than one thousand people using this solution because we were a big, media-based company. If there are many user requests in the front end application and the system is not responding much or has slow performance, the system will automatically scale up the performance hardware requirements. 

I have contacted technical support. I have never faced an issue like that with Denodo. Fortunately, we got some kind of a tutorial PDF, which helps us to deploy everything quickly. 

Before working with Azure, I worked with Python. In the culture I was working in, there was no integration. We were using Pure Python scripting and Python data manipulation tools. For example, we used Python's pandas library, which we coded to transform and orchestrate the data, which is necessary for the endpoint. It was not at all a visual tool. It took more time than Denodo. 

There is no installation because it's on the cloud. You just log on to the cloud with your subscription credentials, then you can use Data Factory directly. 

I implemented through an in-house team. 

Data Factory is very expensive. We are using an Azure subscription, so Data Factory has no direct updates, it's just always the latest version. Compared to Denodo, Azure is very costly. Azure Framework has multiple services, not only Data Factory. So in the cloud-based solution, if you're selecting a particular service, like Data Factory, you need to pay for each request.

I also use Denodo. Data Factory is like a transformation layer, but we need an additional staging database or a data storage facility, which is very expensive compared to implementing Denodo. So we extracted the data using Data Factory, then created a staging database with Azure SQL, which cost a huge amount since it's a physical data area. In Denodo, we just implement a layer, which is all handled in Denodo, and not a physical storage mechanism. I prefer customizable data solutions because they improve performance, creativity, and are helpful for front end people.

In comparison to Data Factory's drag-and-drop interface, Denodo developers need to create all the unified views by coding, so we have to create SQL queries to execute. With Data Factory, you can quickly drag and drop data or tables, but in Denodo, it takes more time because you need to code and test and all that.

I rate Data Factory an eight out of ten, mainly because you need a staging database. I recommend Azure to others, but it depends on architecture. In Data Factory, there is no virtualization environment, no layer of virtualization to help integration and doing caching mechanisms. Though Data Factory is there, Denodo is going further. 

I use this primarily for ETL tasks.

The most valuable feature is the ease in which you can create an ETL pipeline.

The GUI is very helpful when it comes to creating pipelines. The user interface is also very fast.

The connection to Snowflake is easy. I can store data and transform it during the ETL process before sending it to Snowflake.

Azure Data Factory is a bit complicated compared to Informatica. There are a lot of connectors that are missing and there are a lot of instances where I need to create a server and install Integration Runtime.

The support and the documentation can be improved.

There are a lot of tasks that you need to write code for.

I have been using Azure Data Factory for about six months.

I have experience with Informatica and I find it easier to use. For example, there are a lot of connectors that are directly available. Also, Informatica is able to take incremental copies, but with Azure, you have to write code to do that.

I have also worked with Matillion and Fivetran, and I feel that there are a lot of things that Azure can learn from these products. For example, with Fivetran there are very good connectors for copying data between other solutions. This is unlike Azure, where a lot of the time, I have to build my own logic.

The initial setup is complex.

I would rate this solution a seven out of ten.

We use the product for data warehouses. It helps us to load data to warehouses. 

The tool's most valuable features are its connectors. It has many out-of-the-box connectors. We use ADF for ETL processes. Our main use case involves integrating data from various databases, processing it, and loading it into the target database. ADF plays a crucial role in orchestrating these ETL workflows. 

The tool's visual interface is good. The ADS scheduling feature impacts data management by determining when jobs must be run and setting up dependencies. This capability eliminates the need to rely on enterprise data scheduling tools. 

When working with AWS, we have noticed that the difference between ADF and AWS is that AWS is more customer-focused. They're more responsive compared to any other company. ADF is not as good as AWS, but it should be. If AWS is ten out of ten, ADF is around eight out of ten. I think AWS is easier to understand from the GUI perspective compared to ADF.

I have been using the product for 6 months. 

ADF is stable. 

I rate the tool's scalability an eight out of ten. 

The tool's deployment is easy. The deployment typically takes around two to three days to set up. However, the duration may vary depending on factors such as the number of integrated endpoints. In our company, the deployment team had three to four people. This team consisted of an IT engineer, a network engineer, and an ETL admin.

We still haven't required much maintenance since we're still in the development phase. However, as time progresses and we move into production, we'll better understand the maintenance requirements.

ADF is cheaper compared to AWS. 

The tool has met our projects' growing data needs effectively so far. I rate it an eight out of ten.