AWS Lake Formation vs BigQuery comparison

"I can easily move data from cold storage to regular storage."

"AWS Lake Formation works hand in hand with other products."

"The solution has many features that are applicable to events such as audits."

"We use AWS Lake Formation typically for the data warehouse."

"The solution is quite good at handling analytics. It's done a good job at helping us centralize them."

"The most important advantage in using AWS Lake Formation is its ability to connect the data lake to the other technologies in AWS. This is what I advise my clients."

"It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services."

"AWS Lake Formation lets you see all your data and tables on one screen."

"It has a well-structured suite of complimentary tools for data integration and so forth."

"The product’s most valuable feature is its ability to manage the database on the cloud."

"I like that we can synch and run a large query. I also like that we can work with a large amount of data. You don't need to work separately, as it's a ready-made solution. It also comes with a built-in machine-learning feature. Once we start inputting the data, it will suggest some things related to the data, and we can come up with nice dashboards and statistics from a vast amount of data."

"It's similar to a Hadoop cluster, except it's managed by Google."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"The product is serverless. We only need to write SQL queries to analyze the data. We need to pay based on the number of queries. The retrieval time is very less. Even if you write large queries, the tool is able to bring back data in a few seconds."

"One of the most significant advantages lies in the decoupling of storage and compute which allows to independently scale storage and compute resources, with the added benefit of extremely cost-effective storage akin to object storage solutions."

"It's pretty stable. It's fast, and it is able to go through large quantities of data pretty quickly."

"Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance."

"AWS Lake Formation's pricing could be cheaper."

"The solution could make improvements around orchestration and doing some automation stuff on AWS front automation. It would be useful if we could use automation to build images and use hardened images which are CIS compliant."

"For the end-users, it's not as user-friendly as it could be."

"You need to have data experience to use the product."

"If I could improve AWS Lake Formation, I would add more integrations with SageMaker."

"In our experience what could be improved are not the support, performance or monitoring, but at a managerial level, the very expensive professional services of AWS. This could be an area of improvement for them. It's too expensive to acquire their support."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"The primary hurdle in this migration lies in the initial phase of moving substantial volumes of data to cloud-based platforms."

"The price could be better. Compared to competing solutions, BigQuery is expensive. It's only suitable for enterprise customers, not small and medium-sized businesses, as they cannot afford this kind of solution. In the next release, it would be better if they improved their AI bot. Although machine learning and artificial intelligence are doing wonders, there is still a lot of room to enhance them."

"I rate BigQuery six out of 10 for affordability. It could be cheaper."

"For greater flexibility and ease of use, it would be beneficial if BigQuery offered more third-party add-ons and connectors, particularly for databases that don't have built-in integration options."

"Instead of connecting directly to BigQuery, we connect to GCP, Cloud Run, and then to BigQuery, which is a long process."

"We'd like to have more integrations with other technologies."

"When it comes to queries or the code being executed in the data warehouse, the management of this code, like integration with the GitHub repository or the GitLab repository, is kind of complicated, and it's not so direct."

"The product could benefit from improvements in user-friendliness, particularly in terms of the user interface."

"AWS Lake Formation is a bit expensive."

"The price could be better. Usually, you need to buy the license for a year. Whenever you want more, you can subscribe to it, and you can use it. Otherwise, you can terminate the license. You can use it daily or monthly, and we use it based on a project's requirements."

"The product’s pricing could be more flexible for end users."

"The solution is pretty affordable and quite cheap in comparison to PDP or Cloudera."

"Its cost structure operates on a pay-as-you-go model."

"The pricing is adaptable, ensuring that organizations can tailor their usage and costs based on their specific requirements and configurations within the Google Cloud Platform."

"The platform is inexpensive."

"The product operates on a pay-for-use model. Costs include storage and query execution, which can accumulate based on data volume and complexity."

"BigQuery pricing can increase quickly. It's a high-priced solution."