AWS Lake Formation vs BigQuery comparison

"The solution is quite good at handling analytics. It's done a good job at helping us centralize them."

"AWS Lake Formation lets you see all your data and tables on one screen."

"It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services."

"The most valuable features of AWS Lake Formation were the access model itself, as it allows implementation of filters, Blueprints, and row-level and column-level security to mask data that shouldn't be accessed by certain entities, enabling granular control without exposing PII data."

"The solution has many features that are applicable to events such as audits."

"The main benefits that I have seen from using AWS Lake Formation are related to FinOps because you have control of your data and can track your costs since AWS Lake Formation is integrated into a unique platform, which is AWS Cloud Service."

"AWS Lake Formation works hand in hand with other products."

"AWS Lake Formation has several valuable features that enhance data management, and one particularly beneficial aspect is how it facilitates better collaboration within data teams."

"BigQuery excels at structuring data, performing predictions, and conducting insightful analyses and it leverages machine learning and artificial intelligence capabilities, powered by Google's Duarte AI."

"It has a proprietary way of storing and accessing data in its own data store and is 100% managed without you needing to install anything. There is no need to arrange for any infrastructure to be able to use this solution."

"It's straightforward to set up."

"What I like most about BigQuery is that it's fast and flexible. Another advantage of BigQuery is that it's easy to learn."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"The initial setup process is easy."

"BigQuery can be used for any type of company. It has the capability of building applications and storing data. It can be used for OLTP or OLAP. It has many other products within the Google space."

"Its integration with other tools like Atlan through a Google Chrome extension is highly beneficial."

"I haven't seen any measurable benefits from using AWS Lake Formation, such as time saving, resource saving, or efficiency improvements."

"The main challenge we faced with AWS Lake Formation was related to cross-account sharing. Granting access to other AWS accounts for tables or databases in a different AWS account was somewhat difficult."

"You need to have data experience to use the product."

"Information about the pricing, cost, and setup cost of the AWS solutions would be beneficial."

"Rather than creating an additional hundred tools, optimizing a tool to have a centralized location to do governance would be beneficial."

"The solution could make improvements around orchestration and doing some automation stuff on AWS front automation. It would be useful if we could use automation to build images and use hardened images which are CIS compliant."

"If I could improve AWS Lake Formation, I would add more integrations with SageMaker."

"In our current setup, everyone from my business unit uses the same consumer account. When access is requested for a table, everyone using that business unit account receives access. This could present a security concern, though it benefits new team members who automatically receive all necessary access permissions."

"They could enhance the platform's user accessibility."

"I noticed recently it's more expensive now."

"Instead of connecting directly to BigQuery, we connect to GCP, Cloud Run, and then to BigQuery, which is a long process."

"There are areas that could be improved with BigQuery, such as more bolt-on capabilities and the ability to use more bolt-ons for APIs."

"The primary hurdle in this migration lies in the initial phase of moving substantial volumes of data to cloud-based platforms."

"So our challenge in Yemen is convincing many people to go to cloud services."

"An area for improvement in BigQuery is its UI because it's not working very well. Pricing for the solution is also very high."

"When I execute a query, the dashboard doesn't always present the output seamlessly. Troubleshooting requires opening each pipeline individually, which is time-consuming."

"AWS Lake Formation is a bit expensive."

"We are above the free threshold, so we are paying around 40 euros per month for BigQuery."

"The tool has competitive pricing."

"The product operates on a pay-for-use model. Costs include storage and query execution, which can accumulate based on data volume and complexity."

"The solution's pricing is cheaper compared to other solutions."

"BigQuery pricing can increase quickly. It's a high-priced solution."

"Its cost structure operates on a pay-as-you-go model."

"The pricing is adaptable, ensuring that organizations can tailor their usage and costs based on their specific requirements and configurations within the Google Cloud Platform."

"The pricing appears to be competitive for the intended usage scenarios we have in mind."