Azure Data Factory Reviews

Our customers use it for data analytics on a large volume of data. So, they're basically bringing data in from multiple sources, and they are doing ETL extraction, transformation, and loading. Then they do initial analytics, populate a data lake, and after that, they take the data from the data lake into more on-premise complex analytics.

Its version depends on a customer's environment. Sometimes, we use the latest version, and sometimes, we use the previous versions.

It is very modular. It works well. We've used Data Factory and then made calls to libraries outside of Data Factory to do things that it wasn't optimized to do, and it worked really well. It is obviously proprietary in regards to Microsoft created it, but it is pretty easy and direct to bring in outside capabilities into Data Factory.

It is very flexible. You can build any features you want.

There is always room to improve. There should be good examples of use that, of course, customers aren't always willing to share. It is Catch-22. It would help the user base if everybody had really good examples of deployments that worked, but when you ask people to put out their good deployments, which also includes me, you usually got, "No, I'm not going to do that." They don't have enough good examples. Microsoft probably just needs to pay one of their partners to build 20 or 30 examples of functional Data Factories and then share them as a user base.

I have been using this solution for the last five years, but probably, the last three years have been significant.

It has been stable. I have not experienced any issues.

It is decent for most things. I'm not sure if it is necessarily intended for large volume and high-speed streams of data. By large, I mean really big, but for pretty much anything that most users would want to do, including ourselves, it is fine. Our clients are large government organizations.

It scales fine within its environment. You can literally throw another Data Factory in or replicate one and do things pretty quickly. So, it is not at all hard to increase your processing footprint, but you have to pay for it. It doesn't end up being quite expensive. Although I haven't really done it, I would suspect that if I did the equivalent in AWS, Azure would be more expensive than AWS because of the way they price data.

They're all right. I would rate them a seven out of 10. They do fine, but there is a lot that they don't do.

I'm not sure if even Microsoft has enough SMEs from a user point of view. They are helpful for getting it set up, making it work, and helping you figure out why it doesn't work. If you want to ask them about something that you are trying to do, they'll try to direct you to a partner, which is fine, but the partners also don't necessarily have an experience. It is Catch-22. There aren't a lot of people out there with Azure experience because Azure started to be in demand only over the last two years.

The customer used a lot of homebrew stuff. They were doing a lot of internal stuff and some Oracle stuff. They were doing things, and they made a workaround and said, "Okay, we'll bring it into Oracle Database, and then we'll do all these things to it." We're like, "Okay, that works, but then you're taking it out of that database and putting it over into the data lake. I don't understand why are you doing that?" That's what they were doing.

It is pretty straightforward. Devil is in the details, but you can easily get up and running in a day with Data Factory. Anybody who is comfortable in Azure can set up Data Factory, but it takes experience to know what it can and can't do or should and shouldn't do.

It is proven, and it works. Make sure you have a well-defined use case and build a quick prototype to ensure that it, in fact, does what you need. Give yourself some benchmarks. That's exactly what we did. We defined the use case, and then we set up Data Factory. We found a couple of things that it didn't do. We figured out a way to work around those things and have it do those things. After that, we confirmed it. It is operational, and it is doing its job. It has been pretty much error-free since then.

It would become easier to use as more people become Azure-capable. If I want to find an AWS SME, I can get tons. They're expensive, but I have them. If I want to find an Azure SME, I usually have to create them. Azure was later to market than AWS. So, there are fewer people who are experts in Azure, and they are in high demand.

I would rate Azure Data Factory a nine out of 10. They just don't have enough good examples out there of things.

The solution is primarily used for data integration. We are using it for the data pipelines to get data out of the legacy systems and provide it to the Azure SQL Database. We are using the SQL data source providers mainly.

The data pipeline and the orchestration functionality are the most valuable aspects of the solution.

The interface is very good. It seeks to be very responsive and intuitive.

The initial setup is very quick and easy.

I'm more of a general manager. I don't have any insights in terms of missing features or items of that nature.

Integration of data lineage would be a nice feature in terms of DevOps integration. It would make implementation for a company much easier. I'm not sure if that's already available or not. However, that would be a great feature to add if it isn't already there.

We've used the solution for the last 12 months or so.

From what I have witnessed, the solution is quite stable. It doesn't crash or freeze. There are no bugs or glitches. It's reliable.

We work with medium to enterprise-level organizations. Customers have anywhere from 300 employees up to 160,000 employees.

Microsoft offers a great community. There's a lot of support available. We're quite satisfied with the level of assistance on offer.

Since the solution is a service, it's basically just a click and run setup. It's very simple. There's very little implementation necessary. A company should be able to easily arrange it. The deployment doesn't take very long at all.

We do provide the implementation for our clients. We're able to provide templates as well. We have predefined implementation space in Data Factory and provide it to the customer.

While clients might individually evaluate other options, however, we're not aware of that information. I can't say what other solution clients might consider before ultimately choosing Microsoft. I would say that it is likely Talend and maybe SQL Server Integration Services.

We are like an integrator. We are a data warehouse NPI consulting company and we use Data Factory to pull data from different legacy systems and do all these transformations that are necessary in order to provide analytical models.

In our normal scenario is that we are providing Azure SQL Databases together with Azure Data Factory and Power BI. 80% of our customers have recognized such a scenario.

On a scale from one to ten, I'd rate the solution at an eight. We've been largely happy with the capabilities of the product.

It's an integration platform, we migrate data across hybrid environments. We have data in our cloud environment or on-prem system so we use it for when we want to integrate data across different environments. It was a problem for us to get data from different hybrid environments.

From my experience so far, the best feature is the ability to copy data to any environment. We have 100 connectors and we can connect them to the system and copy the data from its respective system to any environment. That is the best feature. 

The user interface could use improvement. It's not a major issue but it's something that can be improved. 

It has the ability to create separate folders to organize objects, Data Factory objects. But any time that we created a folder we were not able to create objects. We had to drag and drop into the folder. There were no default options. It was manual work. We offered their team our feedback and they accepted my request.

I have been using Azure Data Factory for around one year. 

Based on my experience with other products on the market, the stability is good. 

I haven't had much experience with scalability. I know we do have scalability options though. It's used daily. 

There are around 1,000 plus users using this solution in my company. 

It requires two people for maintenance. The administrators are the ones who maintain it and give access to the engineers. They regulate who has privileges. 

We have needed to contact their technical support. If we can't find the answers ourselves on the blogs, we contact them with our questions. We get most of the answers we need from the blogs but if not then we can directly speak to the Microsoft team from the Data Factory interface itself, it's really helpful.

I have only used Data Factory for the cloud. For on-prem we have used SSIS.

The initial setup was a bit complex but once you understand its setup, it's less complex. There are certain processes that need to be followed. Once you understand the process, it becomes easier to implement.

The implementation took a little less than one day. The planning for the deployment takes around one or two days. 

We had a discussion with the Microsoft team about the data. We discussed how we were going to implement. Based on the discussion we were able to deploy. A Microsoft partner helped us with some parts. 

We also evaluated AWS.

The advice that I would give to someone considering this solution is to have some background in data warehousing and ETL concepts. Have the background about data warehousing and ETL that extract, transform, and load. If you have the background you need, you will be successful. If not, then my advice would be to learn a little more about it before using Data Factory.

I would rate Data Factory as an eight out of ten. 

I primarily use Data Factory to ingest data. For example, if we need to pull data into our data warehouse from somewhere like Azure Event Hub or salesforce.com.

We have telemetry that streams into an Azure Event Hub, and Data Factory allowed us to move that data from the Event Hub into our data lake and reduce the cost of that compared to the other tooling we were using.

Data Factory's best features are simplicity and flexibility. It's been very easy to set up connections to different types of data sources to pull data into our warehouse.

Data Factory would be improved if it were a little more configuration-oriented and not so code-oriented and if it had more automated features.

I've been using Data Factory for about three years.

I would rate Data Factory's stability eight out of ten.

I would rate Data Factory's scalability eight out of ten.

The initial setup was straightforward, and only one person was required for deployment.

I would rate Data Factory's pricing nine out of ten.

I think Data Factory is a good fit when you need a light, inexpensive way to ingest data. I would rate it eight out of ten.

We primarily use the solution in a data engineering context for bringing data from source to sink.

The solution is very comfortable to use. I'm happy with the user interface and dashboards. I'm pretty happy with everything about the solution. 

We haven't had any issues connecting it to other products.

It's a stable product. 

I have not found any real shortcomings within the product.

I've been using the solution for the past year. 

The product has been very stable and reliable. I'd rate the stability nine out of ten. There are no bugs or glitches. It doesn't crash or freeze. 

There is a team of 30 people working on the solution. 

I've connected with technical support a few times. 

They sent a support engineer or a field engineer to us, and he helped us out. 

Positive

I'm not sure about the exact cost of the solution. 

I'm a customer and end-user.

Our company chose to use this solution based on the fact that it is a Microsoft product. We're using a lot of solutions, including Outlook and Teams. We also use Power BI. We try to use Microsoft so that everything is under one umbrella. That way, there is no difficulty with connecting anything. 

It's a good solution to use. There are lots of videos available on YouTube, and it is very easy to learn. It's very easy to perform things on it as well, which is one thing that a product like ThoughtSpot lacks. There is no training needed like Power BI. 

I'd rate the solution nine out of ten.

We created data ingestion solutions. We have built interpreters, and we have data factories that pull data from our clients. They submit data via Excel spreadsheets, and we process them into a common homogeneous format.

It has helped with some automation. Instead of individual people reviewing these files, we were able to automate the ingestion process, which saved a bunch of time. It saved hours of repeated manual work.

The data mapping and the ability to systematically derive data are nice features. It worked really well for the solution we had. It is visual, and it did the transformation as we wanted.

I couldn't quite grasp it at first because it has a Microsoft footprint on it. Some of the nomenclature around sync and other things is based on how SSRS or SSIS works, which works fine if you know these products. I didn't know them. So, some of the language and some of the settings were obtuse for me to use. It could be a little difficult if you're coming from the Java or AWS platform, but if you are coming from a Microsoft background, it would be very familiar.

For some of the data, there were some issues with data mapping. Some of the error messages were a little bit foggy. There could be more of a quick start guide or some inline examples. The documentation could be better.

There were some latency and performance issues. The processing time took slightly longer than I was hoping for. I wasn't sure if that was a licensing issue or construction of how we did the product. It wasn't super clear to me why and how those occurred. There was think time between steps. I am not sure if they can reduce the latency there. 

I have been using this solution for a year and a half.

It is very stable.

It is very scalable. It is a cloud product. It is being used by business analysts, business managers, and Azure cloud architects. We have just one developer/integrator for deployment and maintenance purposes.

We have plans to increase its usage. We'll be rolling it out for other clients.

Microsoft has these things well-documented. There were videos. I was able to find answers when I needed them. To the uninitiated, it was a little difficult, but we got there.

It was of medium complexity. Because it goes to the cloud, the duration was short. The deployment was minutes and hours.

We are a consultant and integrator. You can use our company for its implementation.

I would rate this solution a nine out of ten.

My primary use cases for this solution are integration and connecting to the different data stores where we get data and migration activity, deployment, and integrations into using linked services and deployment models.

This solution has allowed me to quickly get analysis, sales data, supply chain data, and eCommerce data.

The most valuable feature of this solution is that it allows more data between on-premises and cloud solutions. It's also useful for orchestration for complex data flows and allows us to do ETL-based transitions heavily. In addition, it allows us to integrate with other third-party systems and compare features and pricing. Other valuable features include database replication, SQL service products, SLA support, data sharing, vendor lock-in, and support for developer tools.

Areas for improvement would be the product's performance and its mapping of data flow. In addition, some of the optimization techniques are not scalable, some naming connections are not supported, and automated testing is not supported in all cases. In the next release, I would like to see support so we can enhance based on the next-level pipelines, writing from scratch, flexible scheduling, and pipeline activity.

I've been using this solution for about a year.

This solution is very reliable.

This solution is scalable.

I am satisfied with the technical support.

I previously worked with Azure SQL database.

The initial setup was complex, but the deployment only took 30 to 40 minutes.

This product is priced at the market standard, which is good given that the product contains all the available assets.

When selecting services, make sure to choose only those you need in order to reduce your costs. I would rate this solution as seven out of ten.

There was a need to bring a lot of CRM and marketing data for some PNL analysis. We are connecting to the Salesforce cloud. In it, there's a specific solution in Salesforce Core CRM for the pharmaceutical industry. We are using the solution to connect to that and we are bringing in the various dimensions and transactions from that data source.

Data Factory itself is great. It's pretty straightforward. You can easily add sources, join and lookup information, etc. The ease of use is pretty good. 

They have a lot of other components like a newer monitor, which helps track and generate alerts for any failed jobs and things of that nature, which is helpful.

At this point in time, they should work on somehow integrating the big data capabilities within it. I have not explored it, but it would be good if somehow we could call a Spark job or something to do with the Spark SQL within ADS so that we wouldn't need a Spark tested outside.

On the UI side, they could make it a little more intuitive in terms of how to add the radius components. Somebody who has been working with tools like Informatica or DataStage gets very used to how the UI looks and feels. 

In ADS, adding a new table or joining a new table and overriding that with an override SQL that I could customize would be helpful.

Being able to debug from the design mode itself would be helpful.

I've been using the solution for one year.

In the latest version, the v2 version, the solution is pretty stable. It does not give unknown letters or things like that.

The solution allows you to create reusable components, so it can be scaled pretty easily.

Being an IT services company, we have a gold or a platinum partnership with Microsoft. For us, getting the technical support we need is not a big issue. Their community is also pretty active in responding to any issues. It's quite good. We've been satisfied with the level of support that is offered.

We were not actually involved in the initial setup. That was all with the client, so I won't be able to comment on it.

In terms of licensing costs, we pay somewhere around S14,000 USD per month. There are some additional costs. For example, we would have to subscribe to some additional computing and for elasticity, but they are minimal. 

For disaster recovery and readability setups, we did that on Data Lake.

We use the public cloud deployment model.

I'd warn others to ensure that the design should be frozen before you start building because overriding each other's code and managing code takes effort. To avoid or to reduce that effort, ensure that the design is frozen. You can build some configurable code rather than hard-coding everything into the jobs. That's the biggest recommendation.

I'd rate the solution seven out of ten. It's a pretty good solution, but over the past year, I've been limited on the number of cases I have on it. If it had a better user interface and was more intuitive I would have rated it higher.