AWS Lake Formation vs BigQuery comparison

"The solution is quite good at handling analytics. It's done a good job at helping us centralize them."

"A favorite feature of AWS Lake Formation is that it provides us with visibility into who has access to a particular table or database in Glue."

"AWS Lake Formation works hand in hand with other products."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"I can easily move data from cold storage to regular storage."

"We use this to reduce latency from minutes to seconds, as we aim for real-time visibility into patient healthcare monitoring."

"The most valuable features of AWS Lake Formation were the access model itself, as it allows implementation of filters, Blueprints, and row-level and column-level security to mask data that shouldn't be accessed by certain entities, enabling granular control without exposing PII data."

"It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services."

"The features I have found most valuable in BigQuery include the query cache, which is good and scales well; it handles huge amounts of data quite well, and the multi-regions feature works well."

"Using BigQuery's central repository brings dispersed information together, which facilitates exploring the data and gaining insights, and consequently, it improves operations, response time, and the business overall."

"The solution's reporting, dashboard, and out-of-the-box capabilities match exactly our requirements."

"The interface is what I find particularly valuable."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"One of the most significant advantages lies in the decoupling of storage and compute which allows to independently scale storage and compute resources, with the added benefit of extremely cost-effective storage akin to object storage solutions."

"We basically used it to store server data and generate reports for enterprise architects. It was a valuable tool for our enterprise design architect."

"The most valuable features of BigQuery is that it supports standard SQL and provides good performance."

"The main challenge we faced with AWS Lake Formation was related to cross-account sharing. Granting access to other AWS accounts for tables or databases in a different AWS account was somewhat difficult."

"Despite following official documentation, configuration problems persisted, requiring weeks of support from multiple AWS engineers to resolve."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"I haven't seen any measurable benefits from using AWS Lake Formation, such as time saving, resource saving, or efficiency improvements."

"AWS Lake Formation's pricing could be cheaper."

"Information about the pricing, cost, and setup cost of the AWS solutions would be beneficial."

"For the end-users, it's not as user-friendly as it could be."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"The primary hurdle in this migration lies in the initial phase of moving substantial volumes of data to cloud-based platforms."

"BigQuery should integrate with other tools, such as Cloud Logging and Local Studio, to enhance its capabilities further and enable powerful and innovative analyses."

"When I open many of the Google Cloud products, I am in an environment that I do not feel familiar with; it is a little overwhelming."

"With other columnar databases like Snowflake, you can actually increase your VM size or increase your machine size, and you can buy more memory and it will start working faster, but that's not available in BigQuery."

"To be very specific, here in the Middle East, I'm based out of the UAE, and Google has a very narrow footprint, a very limited footprint here in the region."

"The solution hinges on Google patterns so continued improvement is important."

"There is a limitation when copying data directly from BigQuery; it only supports up to ten MB when copying data to the clipboard."

"The price could be better. Compared to competing solutions, BigQuery is expensive. It's only suitable for enterprise customers, not small and medium-sized businesses, as they cannot afford this kind of solution. In the next release, it would be better if they improved their AI bot. Although machine learning and artificial intelligence are doing wonders, there is still a lot of room to enhance them."

"AWS Lake Formation is a bit expensive."

"The pricing appears to be competitive for the intended usage scenarios we have in mind."

"The platform is inexpensive."

"The pricing is good and there are no additional costs involved."

"1 TB is free of cost monthly. If you use more than 1 TB a month, then you need to pay 5 dollars extra for each TB."

"The product operates on a pay-for-use model. Costs include storage and query execution, which can accumulate based on data volume and complexity."

"BigQuery is inexpensive."

"BigQuery pricing can increase quickly. It's a high-priced solution."

"The product’s pricing could be more flexible for end users."