AWS Lake Formation vs BigQuery comparison

"It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services."

"AWS Lake Formation works hand in hand with other products."

"The main benefits that I have seen from using AWS Lake Formation are related to FinOps because you have control of your data and can track your costs since AWS Lake Formation is integrated into a unique platform, which is AWS Cloud Service."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"We use AWS Lake Formation typically for the data warehouse."

"We use this to reduce latency from minutes to seconds, as we aim for real-time visibility into patient healthcare monitoring."

"The solution is quite good at handling analytics. It's done a good job at helping us centralize them."

"It stands out in efficiently handling internal actions without the need for manual intervention in tasks like building cubes and defining final dimensions."

"BigQuery excels at data analysis. It processes vast amounts of information using its advanced architecture and sophisticated querying capabilities, making it crucial for critical insights and safe for handling sensitive data."

"Once we moved to BigQuery, we saw ROI in terms of cost savings, with 27% performance improvement in most of our queries and total costs reduced by about 17%."

"The most valuable features of this solution, in my opinion, are speed and performance, as well as cost-effectiveness."

"The query tool is scalable and allows for petabytes of data."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"BigQuery allows for very fast access, and it is efficient in handling large datasets compared to other SQL databases."

"The feature called calibrating the capacity is valuable."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"AWS Lake Formation's pricing could be cheaper."

"For the end-users, it's not as user-friendly as it could be."

"In our experience what could be improved are not the support, performance or monitoring, but at a managerial level, the very expensive professional services of AWS."

"For the end-users, it's not as user-friendly as it could be."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance."

"The solution could make improvements around orchestration and doing some automation stuff on AWS front automation."

"I would like to see version-based implementation and a fallback arrangement for data stored in BigQuery storage. These are some features I'm interested in."

"We'd like to have more integrations with other technologies."

"It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had. In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization."

"There is a good amount of documentation out there, but they're consistently making changes to the platform, and, like, their literature hasn't been updated on some plans."

"There are some limitations in the query latency compared to what it was three years ago."

"The product’s performance could be much faster."

"Instead of connecting directly to BigQuery, we connect to GCP, Cloud Run, and then to BigQuery, which is a long process."

"It can be slower and more problematic compared to other platforms such as Snowflake."

"AWS Lake Formation is a bit expensive."

"We are above the free threshold, so we are paying around 40 euros per month for BigQuery."

"The price could be better. Usually, you need to buy the license for a year. Whenever you want more, you can subscribe to it, and you can use it. Otherwise, you can terminate the license. You can use it daily or monthly, and we use it based on a project's requirements."

"I have tried my own setup using my Gmail ID, and I think it had a $300 limit for free for a new user. That's what Google is offering, and we can register and create a project."

"BigQuery pricing can increase quickly. It's a high-priced solution."

"Price-wise, I think that is very reasonable."

"One terabyte of data costs $20 to $22 per month for storage on BigQuery and $25 on Snowflake. Snowflake is costlier for one terabyte, but BigQuery charges based on how much data is inserted into the tables. BigQuery charges you based on the amount of data that you handle and not the time in which you handle it. This is why the pricing models are different and it becomes a key consideration in the decision of which platform to use."

"The solution's pricing is cheaper compared to other solutions."

"The pricing appears to be competitive for the intended usage scenarios we have in mind."