AWS Lake Formation vs BigQuery comparison

"The most important advantage in using AWS Lake Formation is its ability to connect the data lake to the other technologies in AWS. This is what I advise my clients."

"AWS Lake Formation lets you see all your data and tables on one screen."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"AWS Lake Formation significantly improves the structure of the data mesh, making it superior to previous structures we used."

"The LF-Tag system with granular permissions was key to the project as a functionality of AWS Lake Formation."

"We have observed measurable benefits in terms of cost savings, time savings, resource savings, and efficiency improvements in our workflows."

"The integration of AWS Lake Formation with the IAM for authentication and authorization is very good; I didn't have any problems in the setup and thought it was simple."

"We use AWS Lake Formation typically for the data warehouse."

"It's pretty stable. It's fast, and it is able to go through large quantities of data pretty quickly."

"We basically used it to store server data and generate reports for enterprise architects. It was a valuable tool for our enterprise design architect."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"As a cloud solution, it's easy to set up."

"The solution's reporting, dashboard, and out-of-the-box capabilities match exactly our requirements."

"BigQuery can be used for any type of company. It has the capability of building applications and storing data. It can be used for OLTP or OLAP. It has many other products within the Google space."

"BigQuery processes a substantial amount of data, whether in gigabytes or terabytes, swiftly producing desired data within one or two minutes."

"It's straightforward to set up."

"I think AWS Lake Formation could improve by enforcing the least privilege by design, moving from ad hoc grants to role-based access controls."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"Despite following official documentation, configuration problems persisted, requiring weeks of support from multiple AWS engineers to resolve."

"I would appreciate online support, which I don't have access to in my corporation at the bank, so that is important."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"You need to have data experience to use the product."

"Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"There are areas that could be improved with BigQuery, such as more bolt-on capabilities and the ability to use more bolt-ons for APIs."

"When I open many of the Google Cloud products, I am in an environment that I do not feel familiar with; it is a little overwhelming."

"The price could be better. Compared to competing solutions, BigQuery is expensive. It's only suitable for enterprise customers, not small and medium-sized businesses, as they cannot afford this kind of solution. In the next release, it would be better if they improved their AI bot. Although machine learning and artificial intelligence are doing wonders, there is still a lot of room to enhance them."

"The product could benefit from improvements in user-friendliness, particularly in terms of the user interface."

"When I execute a query, the dashboard doesn't always present the output seamlessly."

"The process of migrating from Datastore to BigQuery should be improved."

"When I open many of the Google Cloud products, I am in an environment that I do not feel familiar with; it is a little overwhelming."

"Some of the queries are complex and difficult to understand."

"AWS Lake Formation is a bit expensive."

"The solution's pricing is cheaper compared to other solutions."

"Its cost structure operates on a pay-as-you-go model."

"The price could be better. Usually, you need to buy the license for a year. Whenever you want more, you can subscribe to it, and you can use it. Otherwise, you can terminate the license. You can use it daily or monthly, and we use it based on a project's requirements."

"The platform is inexpensive."

"I have tried my own setup using my Gmail ID, and I think it had a $300 limit for free for a new user. That's what Google is offering, and we can register and create a project."

"The tool has competitive pricing."

"One terabyte of data costs $20 to $22 per month for storage on BigQuery and $25 on Snowflake. Snowflake is costlier for one terabyte, but BigQuery charges based on how much data is inserted into the tables. BigQuery charges you based on the amount of data that you handle and not the time in which you handle it. This is why the pricing models are different and it becomes a key consideration in the decision of which platform to use."

"The solution is pretty affordable and quite cheap in comparison to PDP or Cloudera."