AWS Lake Formation vs BigQuery comparison

"A favorite feature of AWS Lake Formation is that it provides us with visibility into who has access to a particular table or database in Glue."

"The integration of AWS Lake Formation with the IAM for authentication and authorization is very good; I didn't have any problems in the setup and thought it was simple."

"The most valuable features of AWS Lake Formation were the access model itself, as it allows implementation of filters, Blueprints, and row-level and column-level security to mask data that shouldn't be accessed by certain entities, enabling granular control without exposing PII data."

"The most important advantage in using AWS Lake Formation is its ability to connect the data lake to the other technologies in AWS. This is what I advise my clients."

"We have observed measurable benefits in terms of cost savings, time savings, resource savings, and efficiency improvements in our workflows."

"AWS Lake Formation has several valuable features that enhance data management, and one particularly beneficial aspect is how it facilitates better collaboration within data teams."

"The most important advantage of using AWS Lake Formation is its ability to connect the data lake to the other technologies in AWS."

"The solution has many features that are applicable to events such as audits."

"What I like most about BigQuery is that it's fast and flexible. Another advantage of BigQuery is that it's easy to learn."

"It has a proprietary way of storing and accessing data in its own data store and is 100% managed without you needing to install anything. There is no need to arrange for any infrastructure to be able to use this solution."

"It's pretty stable. It's fast, and it is able to go through large quantities of data pretty quickly."

"The interface is what I find particularly valuable."

"BigQuery is a very nice product; it is a good product, scalable, and has high performance."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"The query tool is scalable and allows for petabytes of data."

"The initial setup is simple."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"I think AWS Lake Formation could improve by enforcing the least privilege by design, moving from ad hoc grants to role-based access controls."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance."

"Information about the pricing, cost, and setup cost of the AWS solutions would be beneficial."

"Athena can be a bit clunky when writing queries, indicating a potential enhancement point for easier user interaction with query tools such as DataGrip using provided driver JARs."

"The main challenge we faced with AWS Lake Formation was related to cross-account sharing. Granting access to other AWS accounts for tables or databases in a different AWS account was somewhat difficult."

"For the end-users, it's not as user-friendly as it could be."

"We'd like to have more integrations with other technologies."

"It would be beneficial to integrate additional tools, particularly from a business intelligence perspective."

"When I open many of the Google Cloud products, I am in an environment that I do not feel familiar with; it is a little overwhelming."

"To be very specific, here in the Middle East, I'm based out of the UAE, and Google has a very narrow footprint, a very limited footprint here in the region."

"It would be helpful if they could provide some dashboards where you can easily view charts and information."

"There is a good amount of documentation out there, but they're consistently making changes to the platform, and, like, their literature hasn't been updated on some plans."

"Some of the queries are complex and difficult to understand."

"I would like to see version-based implementation and a fallback arrangement for data stored in BigQuery storage. These are some features I'm interested in."

"AWS Lake Formation is a bit expensive."

"The pricing appears to be competitive for the intended usage scenarios we have in mind."

"1 TB is free of cost monthly. If you use more than 1 TB a month, then you need to pay 5 dollars extra for each TB."

"The pricing is adaptable, ensuring that organizations can tailor their usage and costs based on their specific requirements and configurations within the Google Cloud Platform."

"The platform is inexpensive."

"The product operates on a pay-for-use model. Costs include storage and query execution, which can accumulate based on data volume and complexity."

"The tool has competitive pricing."

"The product’s pricing could be more flexible for end users."

"We are above the free threshold, so we are paying around 40 euros per month for BigQuery."