AWS Lake Formation vs BigQuery comparison

"AWS Lake Formation works hand in hand with other products."

"The main benefits that I have seen from using AWS Lake Formation are related to FinOps because you have control of your data and can track your costs since AWS Lake Formation is integrated into a unique platform, which is AWS Cloud Service."

"AWS Lake Formation has several valuable features that enhance data management, and one particularly beneficial aspect is how it facilitates better collaboration within data teams."

"The integration of AWS Lake Formation with the IAM for authentication and authorization is very good; I didn't have any problems in the setup and thought it was simple."

"I can easily move data from cold storage to regular storage."

"It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"It's pretty stable. It's fast, and it is able to go through large quantities of data pretty quickly."

"The interface is what I find particularly valuable."

"Its integration with other tools like Atlan through a Google Chrome extension is highly beneficial."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"BigQuery processes a substantial amount of data, whether in gigabytes or terabytes, swiftly producing desired data within one or two minutes."

"It's straightforward to set up."

"As a cloud solution, it's easy to set up."

"We basically used it to store server data and generate reports for enterprise architects. It was a valuable tool for our enterprise design architect."

"In our experience what could be improved are not the support, performance or monitoring, but at a managerial level, the very expensive professional services of AWS. This could be an area of improvement for them. It's too expensive to acquire their support."

"I haven't seen any measurable benefits from using AWS Lake Formation, such as time saving, resource saving, or efficiency improvements."

"For the end-users, it's not as user-friendly as it could be."

"Despite following official documentation, configuration problems persisted, requiring weeks of support from multiple AWS engineers to resolve."

"The main challenge we faced with AWS Lake Formation was related to cross-account sharing. Granting access to other AWS accounts for tables or databases in a different AWS account was somewhat difficult."

"Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"For greater flexibility and ease of use, it would be beneficial if BigQuery offered more third-party add-ons and connectors, particularly for databases that don't have built-in integration options."

"With other columnar databases like Snowflake, you can actually increase your VM size or increase your machine size, and you can buy more memory and it will start working faster, but that's not available in BigQuery. You have to actually open a ticket and then follow it up with Google support."

"Some of the queries are complex and difficult to understand."

"I noticed recently it's more expensive now."

"There are some limitations in the query latency compared to what it was three years ago."

"BigQuery should integrate with other tools, such as Cloud Logging and Local Studio, to enhance its capabilities further and enable powerful and innovative analyses."

"When I open many of the Google Cloud products, I am in an environment that I do not feel familiar with; it is a little overwhelming."

"There is a limitation when copying data directly from BigQuery; it only supports up to ten MB when copying data to the clipboard."

"AWS Lake Formation is a bit expensive."

"The solution's pricing is cheaper compared to other solutions."

"The product operates on a pay-for-use model. Costs include storage and query execution, which can accumulate based on data volume and complexity."

"The pricing appears to be competitive for the intended usage scenarios we have in mind."

"Price-wise, I think that is very reasonable."

"BigQuery is inexpensive."

"The tool has competitive pricing."

"The price is a bit high but the technology is worth it."

"The price could be better. Usually, you need to buy the license for a year. Whenever you want more, you can subscribe to it, and you can use it. Otherwise, you can terminate the license. You can use it daily or monthly, and we use it based on a project's requirements."