Qlik Replicate Reviews

Qlik Compose is basically an integration tool, which has been acquired by Qlik from an Israeli IT company. So that Qlik can become leaders or can jump into the integration space.

So, there are two tools. One is Qlik Replicate, which replicates the whole data. And then after the replication is done, Qlik Compose is primarily designed because if users use Qlik Replicate, it will replicate the data. For example, if users have ERP data in entity relationships, users can offload it instead of building ETL jobs over the same system. Users can offload that kind of load by pulling the data from the Replicate system. Users are not building a data warehouse from that. 

So, Compose will come into the picture from the Qlik stack point of view, which will help users automate it quickly. Users need to define the relationships between different tables of what is present in the OLTP system. Based on that, it will automatically design the dimension. It will automatically design the fact, and it will automatically design the relationships. And it will create the table just like what users do in Erwin. Erwin, it is like users define the relationships and an SQL query is generated. But here, it’s 60% automation, where users define the relationship, and then automatically, it will even identify the dimension. Then, after identifying the dimension attributes and what needs to be in that, it will also generate the call for it. That is the data modeling part. So, the advantage is that data modeling is automated. 

And then users will have something like slowly changing dimensions, later having dimensions, similar pattern jobs in the dimension, and fact ETL process that users need to develop. That, again, is time-consuming. So even that is automated. Usually, that can be automated in Compose.

So overall, what we claim from Qlik Replicate is that 60% of the process can be automated. Users have the data modeling effort, where users manually define relationships and put in some effort, and then it's automated. Even in the ETL process, users manually define some connections, map the attributes together, and specify what they need. After that, the rest is automated. So, 60% of the time is spent there.

Qlik Compose plays a key role after users’ve purchased Qlik Replicate. Once replication is done from one system to another, but the data warehouse isn't in place, that's when users start using Qlik Compose. It pulls in tables from the source system and allows to define relationships between those tables. Once this is done, Compose will automatically create dimensions—this is part of the data modeling process.

After the tables are created, the next step is data integration. Data integration can involve developing jobs, and even data modeling is considered a part of data integration. Essentially, data integration is done from the replicated ERP system, which is an entity-relationship (ER) model. Once the data is replicated, users can decide how to compose the dimensions.

From there, users define the relationships, specify details, and set up ETL jobs. Users might deal with data-driven dimensions, slowly changing dimensions (SCD) types 1, 2, or 3. Users simply drag and drop the source and target tables, and Qlik Compose will automatically generate the required code and complete the integration process.

I haven't implemented this fully myself, but I’ve learned it from a pre-sales perspective and demonstrated it once or twice. If I had implemented it over a year, I would have hands-on experience with everything, but I understand the automation of data modeling and ETL jobs well enough to explain it to customers.

We basically use it for fulfilling our data anayltics purpose. Our client is a major global semiconductor company. We use the tool to prepare data for analysis.

For real-time data integration, we have a setup with our internal configuration matching the client's name. Then there's the usual process... Qlik Replicate will capture those logs using change data capture (CDC) and then send the changes to the target in real-time.

For our source databases, we use SAP DB, AWS, SQL Server, MySQL, and PostgreSQL. Currently, we integrate with Databricks and Snowflake as well. Our main target products are Hadoop, and we're switching to Databricks. So, these are the databases or big data platforms we've integrated using Qlik Replicate.

An example involving the banking project I am currently working on is for a public sector bank in India. Primarily, they are using a database, and the ultra architecture is meant to take a backup of the database and restore it. This process requires downtime, usually done at midnight, resulting in around two hours of inactivity. This leads to inefficient analytics, as they cannot perform real-time analysis.

Since it's a complete backup and restore process, there is a cost associated with this. To address this, I started replicating the data. The transaction is stored in one database, and the information is replicated to another database, which is suitable for analytics. This avoids any load on the transaction system while enabling analytics.

The system is basically user-driven, mostly operating as a local platform. Certain configurations and prerequisites are needed. Collaborative work with the system network and the database administrator to a large extent is necessary to make the most of this tool. 

It supports multiple endpoints. Data retrieved from the system can be pushed to multiple places, supporting various divisions such as marketing, loans, and others. There's no additional cost, and this helps in calculating the ROI of marketing campaigns.

The primary use case is using Qlik Replicate to interface with DB2, allowing for efficient data replication from various sources into a common one or other destinations. It is particularly beneficial for dealing with legacy systems.

Qlik Replicate's ability to correctly map and replicate data, especially when converting complex fields like packed decimal fields into integers, stands out. Its unparalleled capability to handle replication with DB2 is a key feature. Additionally, its change data capture capability and ability to recover data when not available enhances the data integration process.

Qlik Replicate has allowed us to change data capture in legacy systems.

We use Qlik Replicate to change data capture of databases in production environments.

We are transferring our source data to Qlik Replicate.

Qlik Replicate stands out with its cutting-edge technology and its ability to handle diverse data management tasks. This powerful tool allows us to efficiently and swiftly load data into various data stores or destinations, while also enabling easy distribution across different endpoints. A notable feature is its capability to reload data from multiple sources by creating multiple tasks within a brief timeframe of fifteen to twenty minutes. This eliminates the need for manual intervention and ensures quick data loading from different tables.

Furthermore, Qlik Replicate functions as a twenty-four-hour messaging tool, operating as a streaming process. It constantly captures and updates changes within a twenty-four-hour window, allowing for real-time data tracking. The tool utilizes Change Data Capture to capture and store data changes, be it in the source data or metadata. While it may not always apply these changes to the target, it primarily focuses on change processing.

We use the tool as a plugin for CDC. 

The most valuable features of Qlik Replicate are its CDC performance and trigger functions. CDC feature is important to the financial industry. 

Most of the use cases involve changing the data capture and dealing with data replications. The main use case is data replication.

All real-time data migrations are happening through Qlik.

Its capability to replicate the data without impacting the source databases performance-wise is the most useful aspect. In terms of the databases, what they do is they don't actually connect to the database and capture the data, and so they go with the logs. They, handle logs and capture the changes from there, and push those changes in real-time. The delay is in milliseconds. Within milliseconds' time, I can move the data to other databases. It's almost real-time for handling my real-time analytics. 

The solution is stable.

Support has been great.