Enterprise Data Architect at a manufacturing company with 201-500 employees
Real User
2023-08-22T12:03:30Z
Aug 22, 2023
Good morning, Ornit.
We are using the enterprise edition of Hitachi Vantara's Pentaho Data Integration (PDI) (it also has a community edition that is free to use but lacks some of the automation features that the enterprise edition has) for over a decade now and it will do just about anything we throw at it. We use if not only for ETL but also to generate reports that are burst out to the business. I've used Informatica in the past and the thing that impressed me about Pentaho was it was much easier to build transformations. That may have improved since then (this was back in the early 2000s) but having to link each and every field between each step was super tedious. PDI does it by name with the option to change the mapping manually if needed.
Search for a product comparison in Data Integration
I recommend Talend if you're looking for an on-premises solution. For cloud-based options, ADF or Azure Fabric would work well. These tools are suggested based on the diverse data sources and the large data volume involved.
I recommend Maiora's - ZARUS Data Suite for your ETL requirement. With Zarus you can process your data without any codes. It is designed keeping in mind the non-tech persons using an ETL tool.
Kindly share your available time slots to discuss this further.
For those with a
greater focus on the cloud, there is the AWS Athena service, which is
also very interesting and worth studying:
https://aws.amazon.com/athena/?nc1=h_ls
Data Strategist, Cloud Solutions Architect at BiTQ
Real User
2023-08-22T01:20:09Z
Aug 22, 2023
Hi Ornit, My preffered would be Informatica, SSIS, Wherescape. Informatica because it's a mature product that has been outfor a while with minimal development effort required. SSIS for SQL Server based solutions. Wherescape for ETL automation. Wherescape enables SQL users to write etl code in SQL using templates built into the product
Data Integration offers a seamless solution for combining data from different sources, enhancing accessibility and consistency. It is essential for companies looking to use data efficiently, ensuring quick and reliable analysis capabilities.Transforms disparate data systems into unified views, allowing organizations to draw insights and make informed decisions. It supports the demands of modern businesses with technologies that can easily manage and align diverse data formats and...
Good morning, Ornit.
We are using the enterprise edition of Hitachi Vantara's Pentaho Data Integration (PDI) (it also has a community edition that is free to use but lacks some of the automation features that the enterprise edition has) for over a decade now and it will do just about anything we throw at it. We use if not only for ETL but also to generate reports that are burst out to the business. I've used Informatica in the past and the thing that impressed me about Pentaho was it was much easier to build transformations. That may have improved since then (this was back in the early 2000s) but having to link each and every field between each step was super tedious. PDI does it by name with the option to change the mapping manually if needed.
I recommend Talend if you're looking for an on-premises solution. For cloud-based options, ADF or Azure Fabric would work well. These tools are suggested based on the diverse data sources and the large data volume involved.
IBM DataStage. Run all over the world - the only solution that can scale to meet massive data needs with its parallel processing engine.
Hi Ornit,
I recommend Maiora's - ZARUS Data Suite for your ETL requirement. With Zarus you can process your data without any codes. It is designed keeping in mind the non-tech persons using an ETL tool.
Kindly share your available time slots to discuss this further.
Write back to vijayraj.amin@maiora.co
Regards,
Vijayraj Amin
Maiora's ZARUS supports BYOL (Bring your Own License) to connect to your existing systems and applications.
In WSO2 Enterprise
Integrator 7 there is a very interesting and performant ETL
capability that is worth studying:
https://ei.docs.wso2.com/en/latest/streaming-integrator/guides/performing-etl-tasks/
For those with a
greater focus on the cloud, there is the AWS Athena service, which is
also very interesting and worth studying:
https://aws.amazon.com/athena/?nc1=h_ls
Qlik
Hi Ornit, My preffered would be Informatica, SSIS, Wherescape. Informatica because it's a mature product that has been outfor a while with minimal development effort required. SSIS for SQL Server based solutions. Wherescape for ETL automation. Wherescape enables SQL users to write etl code in SQL using templates built into the product
I recommend SSIS especially if you have already the license for MS SQL Enterprise edition