Azure Data Factory Reviews

Azure Data Factory's main use case is to take raw data from the source and move it to the destination like ADLS Gen2. I build end-to-end pipelines with backfilling features in Azure Data Factory. Along with that, I also schedule the pipeline.

I have used Azure Data Factory for copying raw data from a source to a final destination by using drag-and-drop features in a pipeline, as well as for scheduling and orchestration of the pipeline. These are the main features that I have used in Azure Data Factory.

The best features about Azure Data Factory are that it gives the features of drag-and-drop activities. I don't have to write each line for different activities such as copy activity or web activities. I just have to drag and drop those activities and make a pipeline. This is one of the main features. The second feature is that I can schedule my pipeline. Additionally, I can orchestrate my pipelines. If I have a lot of pipelines for a particular job, I need to do the orchestration. Another feature is that it is used to copy the raw data from a source to a destination. These are the main features that Azure Data Factory provides.

Apart from that, as I mentioned, Azure Data Factory is also used for backfilling features. If in my pipeline I am running my pipeline and I come to know that for a particular date my data has not come, I can add the backfilling features in my pipeline. Also, if I have already copied my raw data to the destination and I want to run that pipeline again, I should not have the data that I have already copied come to my destination again. For that, I can use the backfilling features. These are the features I would like to highlight.

Since I have mentioned that I can use the drag-and-drop activities, by using that, Azure Data Factory is saving a lot of time. My organization has already been saving the time and generating more income with that. Second is that I can use the AI features as well in Azure Data Factory because of its built-in AI. These are some of the features in which my organization is, of course, making a profit.

If I were to create a pipeline by writing everything line by line, it would take almost seven to eight days to complete a pipeline. But using the drag-and-drop activities, I can do it in minutes, around 15 to 20 minutes. It is saving almost five to six days of time. Taking a rough example of 6 into 5, it is saving 30 hours of time.

If the AI features were more improved so that I don't have to provide each and every detail, Azure Data Factory could be improved in a much better way by improving the AI features.

For example, if I want to fetch any data from a raw source, I need to provide each and every detail. But if I am just uploading my raw data and if AI will sync with that data, it can analyze that data and give me proper suggestions on how that should be done in a proper way. Automatic suggestions could improve in a much better way.

As I have mentioned, the AI features as well as more drag-and-drop activities could be improved. If I am making a pipeline, it should give me suggestions, such as which activity should be used, so that I don't have to remember each activity. If I have used one activity, I shouldn't have to remember what activity should I use next. It should give auto-suggestions. That is why I have given a nine out of 10.

Currently, I don't know about its governance and security, but in view of its improvement, I think Azure Data Factory should improve in these areas.

As I already mentioned, the AI features should be improved. Also, the auto-suggestion features should also improve.

In terms of accuracy and reliability of its output, it is good. It gives me the accurate result. Reliability is also good. I can rely on it. It is good overall.

Azure Data Factory is good, especially for a beginner because when I was a beginner, I started my journey of data engineering with Azure Data Factory only. When I was learning it, using the drag-and-drop activity made it very simple to learn for a beginner. Additionally, with the help of the AI features, I was testing a lot of things. Also, rescheduling the pipeline and orchestrating the pipeline make it good for a data engineering professional. I gave this product a rating of 9 out of 10.

My use case for Azure Data Factory is mainly for integrating the data for the ETL process.

The most valuable feature I have found at Azure Data Factory is the data flow function.

Regarding the integration feature in Azure Data Factory, the integration part is excellent; we have major source connectors, so we can integrate the data from different data sources and also perform basic transformation while transforming, which is a great feature in Azure Data Factory.

The main benefits users can receive from Azure Data Factory include building reusable common components. We do not want to create a pipeline for each different data set, so we can create a framework to build the model, allowing us to use reusable components for any number of sources based on configuration. This is a great capability in Azure Data Factory.

The orchestration features in Azure Data Factory are definitely useful, as it is not only for Azure Data Factory; we can also include DataBricks and other services for integrating the data solution, making it a very beneficial feature.

Regarding potential areas of improvement for Azure Data Factory, there is a need for better data transformation, especially since many people are now depending on DataBricks more for connectivity and data integration. Azure Data Factory should consider how to enhance integration or filtering for more transformations, such as integrating with Spark clusters.

I am satisfied with Azure Data Factory so far, but I suggest integrating some AI functionality to analyze data during the transition itself, providing insights such as null records, common records, and duplicates without running a separate pipeline or job.

The monitoring tools in Azure Data Factory are helpful for optimizing data pipelines; while the current feature is adequate, they can improve by creating a live dashboard to see the online process, including how much percentage has been completed, which will be very helpful for people who are monitoring the pipeline.

I have been working with Azure Data Factory for more than five to six years.

I would rate the stability of Azure Data Factory as 8.5.

For scalability, I rate Azure Data Factory as eight.

The initial setup for Azure Data Factory is not complex; it is simple, except when using the integration service for on-premises where a separate VM is needed, but it is not complex for people who are technical.

My main competitor in the market for Azure Data Factory would be DataBricks, as they are making many improvements for integrating data sources, and I believe DataBricks will become a big competitor in the future.

I am still working with Azure Data Factory and continue to use the product.

I am satisfied with the built-in connectors for data transformation in Azure Data Factory because there are more than 150 plus and around 200 connectors available now, which means we do not need to use any custom solutions for most of the major data sources.

Azure Data Factory supports hybrid data scenarios, which I find very useful because most sources are moving to the cloud, but some applications remain on-premises or in different clouds, so this hybrid feature is much required.

Azure Data Factory is applicable for all companies, from small to mid-size and enterprise-level use cases.

On a scale of 1-10, I rate Azure Data Factory an 8.

Azure Data Factory is a very easy to use ETL tool for loading and transforming data from one location to another. It has the capability to connect to any instance where heavy lifting can be performed. For example, if you want to extract data, perform heavy lifting, and complete transformation tasks, you can use any compute resource such as Spark-related items or database-related items. Anything that can be connected over a service endpoint can be leveraged. Then, the data can be loaded into a presentable format and published into any visualization tool. It is an end-to-end solution for data engineering work.

In my team at Microsoft, Azure Data Factory was being used. However, in India, the team consisted of only four to five members.

I have used Azure Data Factory for a very long time. Azure Data Factory has the capability to connect to almost all services through the endpoint, similar to the majority of services. It manages threading very well and provides an interactive dashboard where you can navigate data and perform troubleshooting. I appreciate it very much in terms of functionality. I can only compare it with a couple of tools that have similar capabilities, such as Airflow, but I prefer Azure Data Factory.

There is one particular problem with Azure Data Factory. When you have a parent-to-child relationship and the child has one more relationship, creating a hierarchy situation, there are issues. When you want to cancel one particular job recursively to cancel the parent one, that feature does not work very well. Because of this issue, we need to manage some custom scripts to handle this. Apart from that, there are not many issues. Sometimes there are issues with the gateway regarding slowness, but that is also manageable. However, the recursive job cancellation feature is the one part that I think needs to be stabilized more.

I have been using it for eight to nine years.

I have been using Azure Data Factory for a very long time, and I did not find too many issues. There were occasions when there were outages for a particular region, but it has been pretty much stable overall. I did not experience many issues.

I did not experience scalability issues. I had to contact support once for quota increases with Azure Data Factory, but that was resolved within just one interaction. Certain regions and certain services require quota increases that we cannot perform ourselves, so we need to contact the support team and explain the requirement. Apart from that, there were no issues.

I did not use any other solutions for this purpose previously. At that time at OpenAI, Azure Data Factory was integrated with Microsoft, so it was available. However, we did not have any requirement for it because we were handling AI tools-related work from Databricks itself.

I have used Azure Data Factory extensively, even when it was first launched within Microsoft when it was in version one. At that time, we had to write JSON code, and eventually, it became better.

Azure Data Factory is an ETL or ELT tool. With Azure Data Factory, you primarily perform the orchestration part, which is the main function.

It is easy to use because Azure Data Factory is a cloud-based service provided by Microsoft. Your build and release pipeline is very straightforward. You get two files: parameter files and JSON or XML files. You make all these changes and do this configuration with Azure DevOps. It is easy to deploy, so there are not too many complications.

Even though a job runs for ten seconds, the charge will still come for a minute. It is somewhat more costly compared to Workflows, but it is still manageable and not too expensive.

Azure Data Factory is easy to learn for beginners as well because it has proper GUI documentation that is very clear. Microsoft always excels in documentation, so it is good to use.

I would rate this product an eight out of ten.

The main use cases for Azure Data Factory are to integrate data, bringing it from multiple sources into the central enterprise data warehouse from diversified types of data sources.

Azure Data Factory is easy to use and integrates well, providing good classification to bring data from diverse parts of the data infrastructure, whether from CSV files coming from the Cisco server, MySQL, Excel machines, or local network data from remote files. It can schedule the processes and provides almost all comprehensive features of proper integration and scheduling based on our requirements.

From the benefits perspective, I have seen the main benefits of using Azure Data Factory as streamlining the data quality, data cleansing, and data integration for multiple sources.

I would like to see improved speed, performance, and parallel loading in Azure Data Factory. Some jobs that should have finished much earlier took a longer time. Therefore, I see performance as an area of improvement. Additionally, the ability to handle the largest volumes of data is another concern; if I have to manage more than one terabyte of data every day, I am not comfortable dealing with Azure Data Factory and had to switch to Oracle Data Integrators (ODI) because it lacks performance features.

The main features that I would like to see included in Azure Data Factory are a degree of parallelism, the ability to do parallel data delivery, and the capability to pause, terminate, and restart failed batches. I also want to see improvements in the speed and performance of each job.

I have been working with Azure Data Factory for six months since August.

Azure Data Factory is stable.

Azure Data Factory is scalable up to some extent.

I have interfaced with technical support quite a few times and was able to launch my service requests with Microsoft Azure. I was happy with their feedback and answers.

On a scale of one to ten, I would rate the technical support as nine.

The main pros of Azure Data Factory in comparison with Oracle data integration solutions are that it is easily available, easy to use with a proper GUI interface, and lightweight. The cons include the need for better performance options available to manage a more robust track of loading into the warehousing environment.

The initial setup process for Azure Data Factory was straightforward and very simple.

The main pros of Azure Data Factory in comparison with Oracle data integration solutions are that it is easily available, easy to use with a proper GUI interface, and lightweight.

My advice to those who would like to start working with Azure Data Factory is that if you are looking for a good tool to facilitate loading data within a reasonable volume of structured data, Azure Data Factory is a very good tool. It is easy to use and has almost all the features required for data loading. However, if you anticipate dealing with certain data volumes, such as complexities and volumes exceeding two terabytes per day or over two single reports a day, Azure Data Factory might struggle. You may need a specific infrastructure size to support those functionalities to manage expectations properly. Therefore, if you are handling medium-scale to semi-enterprise data, you need to design the architecture carefully to maximize the tool's effectiveness. I would rate this product overall as a seven out of ten.

I usually recommend Azure Data Factory for finance companies, particularly in finance, and in some cases for retail instead of Oracle.

It is mostly suitable for mid-size, mid-enterprise, and big enterprise companies.

The orchestrating feature in Azure Data Factory is quite good, and I have nothing to complain about. We were using it in a few projects and everything works well.

I would assess the built-in connectors for data transformation in Azure Data Factory as quite advanced in functionality, and I'm happy with it.

Azure Data Factory's data flows are quite good in handling large data volumes, rating them around eight or nine out of ten.

The initial setup of Azure Data Factory is straightforward and not complicated; it works quite simply.

I value the integration capabilities with other platforms and software in Azure Data Factory the most.

The monitoring tools in Azure Data Factory have helped optimize the data pipelines, though it's a bit tricky to track errors, so I can't say that it's brilliant, but it works.

Azure Data Factory's hybrid data scenario support is supported quite well.

There is room for improvement in Azure Data Factory's monitoring tools, as I think that bug tracking and error investigation could be improved because it's difficult to retrieve details for some cases.

Azure Data Factory could be improved in terms of monitoring, specifically in error troubleshooting and monitoring aspects.

I am expecting some automation for routine operations and tasks that currently require manual configuration in the next release of Azure Data Factory.

They should work on optimizing their licensing model and pricing structure. I'm not very frequently working with the pricing given that it's handled by separate people, but overall I previously faced some issues during the investigation of costs and pricing for customers. It very much depends on what licenses for which Microsoft products a client already has, so in many cases it's very difficult to come up with even a general understanding of what expenses a client can have with this product. Microsoft always asks when you're inquiring about the price, "What do you have already?" So it's very difficult.

The pricing of Azure Data Factory is reasonable for small and medium-sized companies considering their usage, but it is expensive for enterprise.

I have been working with Azure Data Factory for about five years.

I currently work with both Amazon and Azure products. I'm working with a database on the Amazon platform and additionally have some tasks on Azure Cloud.

The initial setup of Azure Data Factory is straightforward and not complicated; it works quite simply.

I'm currently working with Power BI and Azure Data Factory in Azure Cloud.

I work as a consultant for a software services provider company when it comes to Azure Data Factory.

I gave this product a rating of eight out of ten.

I am using Azure Data Factory, but only in a small area, not on a large scale.

I have an account with PeerSpot and have downloaded reports on Microsoft Azure and Oracle solutions; I am currently working with Azure. Oracle and perhaps Informatica are similar products.

Azure Data Factory was not difficult to deploy because it is a small area, so we completed it very quickly.

I have been using Azure Data Factory for two years.

I am not the technical person, so I cannot tell you if I am using the orchestration feature of Azure Data Factory.

As an end user, I cannot provide feedback on improvements for the software because I am not in the technical or management roles; I am using the environment.

I think more AI features could be added to Azure Data Factory in the future.

I have been using Azure Data Factory for two years.

I have not used the monitoring feature or the monitoring tools of Azure Data Factory.

I did not see any positive impact from Azure Data Factory overall.

I am ready to rate the technical support of the product.

I think the technical support of Microsoft is a six.

They are not slow on responding or very informative.

Positive

Azure Data Factory was not difficult to deploy because it is a small area, so we completed it very quickly.

We completed the deployment quickly, like an hour or a couple of hours, because it is a test environment for our lab for our use.

We did the installation all by ourselves.

We did the installation all by ourselves.

I did not see any positive impact from Azure Data Factory overall.

I think Azure Data Factory deserves a rating of seven or maybe eight.

I am a solution architect and work with the presales team. I generally provide solutions, study Azure Data Factory, and conduct POCs. I use Azure Data Factory as an ETL tool. I have been using it for about three years.

Azure Data Factory is a low code, no code platform, which is helpful. It provides many prebuilt functionalities that assist in building data pipelines. 

Also, it facilitates easy transformation with all required functionalities for analytics. Furthermore, it connects to different sources out-of-the-box, making integration much easier. The monitoring is very thorough, though a more readable version would be appreciable.

There are performance issues, particularly with the underlying compute, which should be configurable. Sometimes, the compute fails to process data if there is a heavy load suddenly, and it doesn't scale up automatically. Improvements in data governance and data lineage features would also be appreciated.

I have been using it for about three years.

The stability is rated seven. Sometimes, Microsoft updates services, and backward compatibility is absent. Occasionally, pipeline failures occur due to the compute issues mentioned earlier.

It is scalable. However, sometimes there are problems, however, they are resolvable.

The technical support is responsive and helpful, rated nine out of ten.

Positive

We build pipelines for clients who benefit when moving from on-prem systems to cloud, often resulting in cost reductions.

The licensing cost is rated seven out of ten. It is considered cost-effective.

I work mostly with Azure and a little with AWS or GCP.

The overall rating for Azure Data Factory is eight out of ten. It meets most of my requirements. I recommend it as a cost-effective and quick solution for building data pipelines and integrating with various sources.

We use Azure Data Factory to perform transformation operations after moving away from a legacy ETL platform. The platform enables us to handle a large number of ETL jobs and integrate with other Azure services for improved automation and scalability. It supports advanced parsing of structured data formats like XML and JSON and integrates well with our reporting and analytics systems.

Data flow operations are intuitive, particularly for applying transformations and using expressions.

The platform makes it easy to preview and manipulate data during development.

Direct support for XML and JSON datasets improves processing efficiency.

Integration with Azure services supports our cloud-first approach.

Cluster startup times can introduce a delay and affect cost when not optimized.

Integration with Git services needs better support to handle environment-specific configurations smoothly.

Enhanced support for hybrid integration scenarios, like on-premise servers, would be valuable.

I have been using Azure Data Factory for around 2 years.

In comparison to our previous solution, Azure Data Factory operates at approximately 30% of the former cost. Our stakeholders and clients have expressed satisfaction with its efficiency and cost-effectiveness.

We've seen a significant reduction in our data integration costs, with Azure Data Factory operating at a fraction of our previous solution’s cost. It has improved development agility and operational efficiency.

Azure Data Factory offers a modern and cost-effective approach to building data pipelines, especially for teams transitioning to cloud-native solutions. It provides a user-friendly interface and robust transformation tools. For organizations planning ETL modernization, ADF is a strong candidate with solid ecosystem support.

Rating: 9/10

We are a partner and operate as a reseller. Our main area is as a service provider, working with businesses in building their data pipelines, data engineering, and data analytics solutions. We often work with clients on their infrastructure, with Data Factory being a key component more commonly in these situations.

The interface of Azure Data Factory is very usable with a more interactive visual experience, making it easier for people who are not as experienced in coding to work with. The scheduling management of pipelines and how the environment is set up allows for effective troubleshooting and identifying issues quickly. Alerts and warnings make it easy to diagnose pipeline errors.

Some prebuilt data source or data connection aspects are generic. In particular, sourcing data through APIs could be improved. Currently, standard API connector tools are limited, so incorporating more dedicated API sources to specific services like HubSpot CRM or Salesforce would be beneficial. These require some customization and development currently.

I have approximately five or six years of experience working with Azure Data Factory.

The solution has a high level of stability, roughly a nine out of ten. While there have been small outages, nothing substantial has impacted our operations.

Azure Data Factory is highly scalable. There is no real limit, and it can handle high demand and workloads. The limitation is more dependent on the systems it connects to, such as databases.

The technical support from Microsoft is rated an eight out of ten. We had to contact them multiple times over a few years. While they did not always resolve issues quickly, they supported us reasonably well in other cases.

Positive

The initial setup could be rated as a nine out of ten. With a good knowledge base in data transformation and management, our team found it a very usable tool, not necessarily easy but achievable for successful builds.

The pricing is cost-effective. Azure Data Factory itself bears a low cost since it mainly manages scheduling and job initiation. The real cost emerges in connected systems or databases when running high-frequency tasks with large data volumes.

Based on my experience, I would recommend the solution to others. 

I would rate the overall solution as an eight or nine out of ten.

I find the most valuable feature in Azure Data Factory to be its ability to handle large datasets. The data is more scalable.

There is a problem with the integration with third-party solutions, particularly with SAP.

The scalability of Azure Data Factory is good. I find the data more scalable.

I faced issues that were overcome satisfactorily. The technical support for Azure Data Factory is generally acceptable.

The pricing of Azure Data Factory is another aspect to consider. Although no specific figures were given, it was a point of discussion.

Scalability and price were mentioned as advantages that Databricks has over Azure Data Factory.

Azure Data Factory helps with AI initiatives. I would still recommend Azure Data Factory to others. For the next release, I would like to see additional features. I'd rate the solution seven out of ten.