Apache Airflow Reviews

Name: Apache Airflow
Brand: Apache
Rating: 4.0 (36 reviews)

4.0 out of 5

36 reviews
97% willing to recommend

713 followers

What is Apache Airflow?

Apache Airflow is an open-source workflow management system (WMS) that is primarily used to programmatically author, orchestrate, schedule, and monitor data pipelines as well as workflows. The solution makes it possible for you to manage your data pipelines by authoring workflows as directed acyclic graphs (DAGs) of tasks. By using Apache Airflow, you can orchestrate data pipelines over object stores and data warehouses, run workflows that are not data-related, and can also create and manage scripted data pipelines as code (Python).

Get the Apache Airflow Buyer's Guide and find out what your peers are saying about Apache Airflow, Informatica Intelligent Data Management Cloud (IDMC), Camunda and more!

Apache Airflow is the #2 ranked solution in BPM Software. PeerSpot users give Apache Airflow an average rating of 8.0 out of 10. Apache Airflow is most commonly compared to Informatica Intelligent Data Management Cloud (IDMC): Apache Airflow vs Informatica Intelligent Data Management Cloud (IDMC). Apache Airflow is popular among the large enterprise segment, accounting for 73% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a financial services firm, accounting for 25% of all views.

Helped 815,854 peers since 2012

Featured reviews

Sanket Suhagiya

Senior Data Engineer at LTIMindtree

The primary use case for us is ETL pipelines. We write some pipelines to ingest the data. That is the primary use. And, secondly, we use it to run some scheduling and orchestration. We need to run some automation jobs every day. So, we just write an Airflow task and pipeline that runs every day or…

Read full review

Damian Bukowski

Program Python at Santander Bank Polska

The only thing I would like Apache to do is to introduce an integration of the database from Oracle because it currently supports Postgres primarily in MySQL. Oracle is something that many companies use, like a production database, for which you have to pay since it is not free and offers more extended support. With Apache Airflow, even though it uses Python and Python has modules that include Oracle databases, it'll be safer and more convenient to do it through Apache Airflow and not through Python scripts. I want to see Apache Airflow have more integrations with more production-based databases since it is an area where the product lacks currently.

Read full review

SUDHIR KUMAR RATHLAVATH

Student at University of South Florida

One improvement could be the inclusion of a plugin with a drag-and-drop feature. This graphical feature would be beneficial when dealing with connectivity and integration services like connecting to BigQuery or other systems. As a first-time user, although the documentation is available, it would be more user-friendly to have a drag-and-drop interface within the portal. Users could simply drag and drop components to create a pseudo-code, making it more flexible and intuitive. Therefore, I suggest having a drag-and-drop feature for a more user-friendly experience and better code management. Moreover, for admins, there should be improved logging capabilities because Apache Airflow does have logging, but it's limited to some database data. It would be better if everything goes into the server where it's hosted. Probably on the interface level. If something goes well for the developers.

Read full review

Apache Airflow mindshare

As of November 2024, the mindshare of Apache Airflow in the Business Process Management (BPM) category stands at 7.2%, down from 9.3% compared to the previous year, according to calculations based on PeerSpot user engagement data.

Business Process Management (BPM)

Key learnings from peers

Last updated Oct 2, 2024

Valuable Features

"The core features are strong, which are supported by Apache Airflow variables, DAGs, and connections."
"Airflow integrates well with Cloudera and effectively supports complex operations."
"I worked on a project at a leading German bank for two years, successfully migrating large applications with hundreds of jobs."

Room for Improvement

"The UI is a little bit outdated according to modern standards."
"One area for improvement would be to address specific functionalities removed in recent updates that were previously useful for our operations."
"Apache Airflow improved workflow efficiency, but we had to find solutions for large workflows. For instance, a monthly workflow with 1200 jobs had to be split into three to four pieces as it struggled with large job numbers. Loading a workflow with 500 jobs could take 10 minutes, which wasn't acceptable."

Pricing

"Apache Airflow is open-source and free. Hyperscalers like Google (with Composer), Azure, and AWS offer managed Airflow services."
"I use the tool's open-source version."
"It is an open-source tool. There are no additional fees or charges associated with the product."

These insights are based on the in-depth reviews provided by peers to help you make a better buying decision.

Download our Apache Airflow Buyer's Guide for additional reliable information.

Review data by company size

By reviewers

By visitors reading reviews

Top industries

By visitors reading reviews

Financial Services Firm

25%

Computer Software Company

14%

Manufacturing Company

Retailer

Healthcare Company

Insurance Company

Educational Organization

Government

Energy/Utilities Company

Comms Service Provider

University

Media Company

Real Estate/Law Firm

Construction Company

Non Profit

Hospitality Company

Recreational Facilities/Services Company

Pharma/Biotech Company

Logistics Company

Transportation Company

Wholesaler/Distributor

Legal Firm

Outsourcing Company

Consumer Goods Company

Performing Arts

Aerospace/Defense Firm

Compare Apache Airflow with alternative products

Learn more about Apache Airflow

Apache Airflow Features

Apache Airflow has many valuable key features. Some of the most useful ones include:

Smart sensor: In Apache Airflow, tasks are executed sequentially. The smart sensors are executed in bundles, and therefore consume fewer resources.

Dockerfile: By using Apache Airflow’s dockerfile feature, you can run your business’s Airflow code without having to document and automate the process of running Airflow on a server.

Scalability: Because Apache Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers, you can easily scale it.

Plug-and-play operators: With Apache Airflow, you can choose from several plug-and-play operators that are ready to execute your tasks on many third-party services.

Apache Airflow Benefits

There are many benefits to implementing Apache Airflow. Some of the biggest advantages the solution offers include:

User friendly: Using Apache Airflow requires minimal python knowledge to get started.

Intuitive user interface: The Apache Airflow user interface enables you to visualize pipelines running in production, monitor progress, and also troubleshoot issues when needed.

Easy integration: Apache Airflow can easily be integrated with cloud platforms (Google, AWS, Azure, etc).

Visual DAGs: Apache Airflow’s visual DAGs provide data lineage, which facilitates debugging of data flows and also aids in auditing and data governance.

Flexibility: Apache Airflow provides you with several ways to make DAG objects more flexible. At runtime, a context variable is passed to each workflow execution, which is quickly incorporated into an SQL statement that includes the run ID, execution date, and last and next run times.

Multiple deployment options: With Apache Airflow, you have several options for deployment, including self-service, open source, or a managed service.

Several data source connections: Apache Airflow can connect to a variety of data sources, including APIs, databases, data warehouses, and more.

Reviews from Real Users

Below are some reviews and helpful feedback written by PeerSpot users currently using the Apache Airflow solution.

A Senior Solutions Architect/Software Architect says, “The product integrates well with other pipelines and solutions. The ease of building different processes is very valuable to us. The difference between Kafka and Airflow, is that it's better for dealing with the specific flows that we want to do some transformation. It's very easy to create flows.”

An Assistant Manager at a comms service provider mentions, “The best part of Airflow is its direct support for Python, especially because Python is so important for data science, engineering, and design. This makes the programmatic aspect of our work easy for us, and it means we can automate a lot.”

A Senior Software Engineer at a pharma/biotech company comments that he likes Apache Airflow because it is “Feature rich, open-source, and good for building data pipelines.”

Apache Airflow was previously known as Airflow.