AWS Lake Formation vs BigQuery comparison

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"The most valuable features of AWS Lake Formation were the access model itself, as it allows implementation of filters, Blueprints, and row-level and column-level security to mask data that shouldn't be accessed by certain entities, enabling granular control without exposing PII data."

"AWS Lake Formation has several valuable features that enhance data management, and one particularly beneficial aspect is how it facilitates better collaboration within data teams."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"AWS Lake Formation lets you see all your data and tables on one screen."

"We use this to reduce latency from minutes to seconds, as we aim for real-time visibility into patient healthcare monitoring."

"It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services."

"In the shortest form, what I appreciated about AWS Lake Formation was that the schema definition and data cataloging were quite good."

"The most valuable features of this solution, in my opinion, are speed and performance, as well as cost-effectiveness."

"The query tool is scalable and allows for petabytes of data."

"The initial setup is simple."

"The initial setup process is easy."

"When integrating their system into the cloud-based solutions, we were able to increase their efficiency and overall productivity twice compared with their on-premises option."

"The interface is what I find particularly valuable."

"As a cloud solution, it's easy to set up."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"I would appreciate online support, which I don't have access to in my corporation at the bank, so that is important."

"Athena can be a bit clunky when writing queries, indicating a potential enhancement point for easier user interaction with query tools such as DataGrip using provided driver JARs."

"The solution could make improvements around orchestration and doing some automation stuff on AWS front automation. It would be useful if we could use automation to build images and use hardened images which are CIS compliant."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"I haven't seen any measurable benefits from using AWS Lake Formation, such as time saving, resource saving, or efficiency improvements."

"You need to have data experience to use the product."

"If I could improve AWS Lake Formation, I would add more integrations with SageMaker."

"For the end-users, it's not as user-friendly as it could be."

"As a product, BigQuery still requires a lot of maturity to accommodate other use cases and to be widely acceptable across other organizations."

"The main challenges are in the areas of performance and cost optimizations."

"When I execute a query, the dashboard doesn't always present the output seamlessly."

"We'd like to have more integrations with other technologies."

"It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had. In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization."

"When I open many of the Google Cloud products, I am in an environment that I do not feel familiar with; it is a little overwhelming."

"Sometimes, support specialists might not have enough experience or business understanding, which can be an issue."

"They could enhance the platform's user accessibility."

"AWS Lake Formation is a bit expensive."

"The solution's pricing is cheaper compared to other solutions."

"The tool has competitive pricing."

"The pricing appears to be competitive for the intended usage scenarios we have in mind."

"The solution is pretty affordable and quite cheap in comparison to PDP or Cloudera."

"The pricing is adaptable, ensuring that organizations can tailor their usage and costs based on their specific requirements and configurations within the Google Cloud Platform."

"Price-wise, I think that is very reasonable."

"The platform is inexpensive."

"The price is a bit high but the technology is worth it."