BigQuery Reviews

BigQuery allows you to quickly analyze logs from your systems to identify the severity of issues. It integrates well with other Google Cloud services, such as Cloud Logging, where you can easily manipulate various data types and analyze all logs.

BigQuery excels at data analysis. It processes vast amounts of information using its advanced architecture and sophisticated querying capabilities, making it crucial for critical insights and safe for handling sensitive data. 

BigQuery should integrate with other tools, such as Cloud Logging and Local Studio, to enhance its capabilities further and enable powerful and innovative analyses.

I have been using BigQuery for two years.

I have opted for Fireye, Elasticsearch, and Alcon. One principal difference is that BigQuery starts with machine learning and WAN implementations, while you can implement VMware or other active boxes. Therefore, it is recommended that cloud VMs be used for BigQuery processes. You can execute jobs in the cloud, such as VMware.

For instance, you can compute analytics for email, apply filters, and manipulate weather data. It provides higher efficiency, though exact benchmarks are unclear. Additionally, starting the query flow login request can also be advantageous.

The initial setup is automatic. It requires one person. You need to log in to the Google Cloud platform, import the necessary package into your query, and then you can start querying your data. 

If you need a solid CRM solution integrated with Azure, you'll need knowledgeable people to support it. Three individuals can form a strong CRM team connected to Azure, leveraging BigQuery.

You can use BigQuery to generate and manage large datasets efficiently. Whether using a flexible integrated environment like Dataflow or a local studio, BigQuery provides powerful tools for querying and analyzing data.

The product is free of cost.

Setting up BigQuery on GCP is crucial. When creating a service account, you define the permissions required for project identification or access monitoring systems.

You configure policies using IAM roles to manage access permissions effectively within GCP. These roles govern the service accounts created for specific tasks such as data processing, system monitoring, or other service integrations. When you activate these policies, a JSON token is generated. This token can authenticate and authorize access to Google services like BigQuery or other third-party applications.

Moreover, by configuring VMs to match data processing requirements, you ensure that the data is securely handled by the applications associated with the service accounts. This setup enables seamless communication between your applications and Google services, facilitating efficient data acquisition and processing.

Overall, I rate the solution a seven out of ten.

We basically used it to store server data and generate reports for enterprise architects. It was a valuable tool for our enterprise design architect.

Everything they advertised or listed worked exactly as promised. That was advantageous to us. 

In future releases, I would like to see more pre-defined aggregated forms. After using BigQuery, we need to use the data in an enterprise architecture dimensional data model. So, having pre-defined aggregated forms would be helpful. 

Additionally, I would like to see version-based implementation and a fallback arrangement for data stored in BigQuery storage. These are some features I'm interested in.

I have experience with BigQuery. 

When I joined the company, BigQuery was already implemented by our team.

It is a cheap solution. 

I would recommend getting a clear understanding of BigQuery's functionalities and what it's best suited for. If your needs align with its capabilities, then you should definitely proceed. 

BigQuery offers fantastic features, but it's important to understand its purpose beforehand. Otherwise, you might face difficulties later on.

Overall, I would rate the solution an eight out of ten.

We are primarily using the solution to crunch data. Then, we are doing some ETL work on top of the data. 

We like that it is a serverless system. 

We can analyze terabytes of data in a very small amount of time. 

It's similar to a Hadoop cluster, except it's managed by Google.

The initial setup is simple.

We find the product to be very stable.

It scales quite well.

If they can provide any charting platform on top of this product, that would be ideal. BigQuery now only allows us to run queries. It doesn't provide us with any insights. For example, if a query took so many times, they could maybe provide any suggestions on how to optimize the queries or speed up the process. It would be helpful if they could provide some dashboards where you can easily view charts and information. That would be very useful.

I've been using the solution for two or three years. 

This is a highly stable product. There are no bugs or glitches. It doesn't crash or freeze. 

The solution is very scalable. 

Almost my entire team uses it. We have a 50-member team, and pretty much everyone is on it. They are mostly data engineers and developers. 

We have yet to reach out to technical support. We haven't had any issues. 

We chose this solution specifically since all of our services are in GCP, Google Cloud. Google Cloud has a basic internal coupling with BigQuery. That's the reason we are using BigQuery.

The initial setup is very easy. You just have to log in to the Google Cloud console, and then you can just create a few tables and start using it. 

From start to finish it takes about half an hour. It is even less than that to get the tables up and running. The deployment is quite fast.

I'm not sure about the exact cost, however, it is charged on the queries which you run, basically. For example, if you run a query, the amount of data scanned through BigQuery will dictate the costs. 

I am a customer and end-user.

I'm not sure which version of the solution we're using. 

It's a serverless platform deployed on a public cloud. 

I'd advise potential users to set up their tables accordingly. There are two sets of optimization that BigQuery provides as well. You set up whichever columns you want to do the partition and on which columns you want to do the clustering. If these columns are defined properly, then BigQuery's a breeze to use.

On a scale from one to ten, I would rate it at an eight. If they just added a few more features, it would be almost perfect.

We use BigQuery to store data in a table and query it. Data storage can be either an internal native table or an external table where the external source will point to Google Cloud Storage or Google Drive. 

Wherever we can have external storage, we can have a table built pointing to that external storage and query the tables. In BigQuery, we can query the table or even do DML operations, like insert, delete, etc.

The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. 

The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting.

I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers.

Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. 

It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. 

To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted.

For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage.

It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. 

We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had.

In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization.

I have been using BigQuery for more than three years.

BigQuery is a stable solution.

BigQuery is highly scalable. We can have unlimited storage if we do 20 records, and It's very fast. Even if we scale it to 20 trillion, it will still be fast. 

In my organization, about two in five use BigQuery. When I joined the company a year back, usage was relatively moderate. However, now usage increased because of the on-premise to GCP migration. Because of many successful projects, several people are using BigQuery now.

We have dedicated support people who help us with the framework. If there is a technical issue in BigQuery, we just get help from the technical team. But if there are any engineering issues or some data issues, our team will handle them.

I use Teradata and then Apache Spark on-premises.

The initial setup is relatively straightforward. There are some restrictions, like the project's name. It has to be unique, but once that project is created, we can simply go to an option, query, and the query control will open, and we can start creating a table, loading data, querying, and everything. So that's quite simple and straightforward.

When I joined PayPal, the setup was done in-house. When I worked at another organization, Cognizant, we had Google's help. So a Google specialist helped us set up and everything.

I have tried my own setup using my Gmail ID, and I think it had a $300 limit for free for a new user. That's what Google is offering, and we can register and create a project.

On a scale from one to ten, I would give BigQuery an eight.

It is a pivotal component in enterprise data architecture, and crucial in data lake operations, whether supporting data warehouses or functioning as part of a broader data lake ecosystem.

One of the most significant advantages lies in the decoupling of storage and compute which allows to independently scale storage and compute resources, with the added benefit of extremely cost-effective storage akin to object storage solutions. Its unique architecture not only provides robust enterprise data warehouse capabilities but also seamlessly integrates with data lake functionalities.

The main challenges are in the areas of performance and cost optimizations. Achieving optimal results demands a certain level of familiarity with the platform's internals. The key point for improvement lies in the performance optimization.

I have been working with it for three months.

It exhibits a high level of stability and security, there are no notable issues in these aspects. I would rate it nine out of ten.

It is designed to seamlessly scale with the growing demands of data processing, there are no issues with it. I would rate it nine out of ten.

The technical support is commendable. However, there is room for improvement in the availability of resources and documentation from a technological standpoint. I would rate it seven out of ten.

Neutral

In the landscape of enterprise data warehouses, BigQuery stands out as a superior choice when compared to alternatives like Azure Synapse, AWS Redshift, and Snowflake. While Snowflake is known for its higher costs, and Redshift is perceived as both complex and expensive, Azure Synapse presents its own set of constraints with its MPP architecture and reliance on an RDBMS in-between. BigQuery, on the other hand, has a distinct edge with its seamless migration process, vast capabilities, and a harmonious balance of storage, computing, cost-effectiveness, and performance efficiency. This is particularly evident as organizations and professionals, including myself, have experienced ease in migrating from other vendors to BigQuery. Drawing from my extensive experience working across various cloud platforms such as AWS, Azure, and Snowflake, BigQuery consistently emerges as a robust and preferable solution.

The initial setup is straightforward.

Its cost structure operates on a pay-as-you-go model. I would rate it seven out of ten.

Whether for small, medium, or large enterprises, it is a recommendable choice. Its pricing model makes it accessible and manageable based on your usage. Given that many individuals and businesses already have Gmail accounts and utilize Google Cloud workspaces, incorporating BigQuery into operations is seamless. Moreover, a complimentary reporting tool, Looker Studio, is available for free, enhancing the reporting capabilities on BigQuery or via Google Sheets. Overall, I would rate it eight out of ten.

We use BigQuery at our organization to access daily transactional data from our POS solutions, which are used to sell products to our clients. We gather the most essential information for our clients and upload it to our data lake using BigQuery.

We gather the most essential information for our clients and upload it to our data lake using BigQuery. At the end of the month, we have sufficient information in our data lake to generate legal reports, balances, and reconciliations with partners.

The most valuable aspect of BigQuery is its ability to handle high data workloads without causing friction with our online systems. We can obtain significant amounts of data, which is critical, even if it's not in real-time. 

Additionally, we can solve small issues while working with the platform, and it's rare that we need external support.

Sometimes, support specialists might not have enough experience or business understanding, which can be an issue. They might have basic knowledge but lack specific insights related to the specific configuration or context required by the client.

Google's customer service is good but not the best. They receive a score of eight out of ten.

Positive

Setting up BigQuery is not difficult. Although I do not directly handle this aspect, my team appears comfortable with it and does not encounter major issues requiring outside assistance.

I rate BigQuery nine out of ten. I recommend it to others and have used it in various situations over the years.

BigQuery is a powerful tool for managing and analyzing large datasets. The versatility of BigQuery extends to its compatibility with external data visualization tools like Power BI and Tableau. This means you not only get query results but can also seamlessly integrate and visualize your data for better insights.

The product's most valuable feature is its ability to connect to visualization tools. 

Some of the queries are complex and difficult to understand. 

I have been using the product for more than a year. 

My company has 100 users for BigQuery. 

The tool's support is fast to respond. 

Positive

The tool's deployment is easy if you follow Google's documentation. 

If you have a big database and lots of analytics, BigQuery is a really good tool. It helps save and manage your queries and gives you results you can show clients and others. I rate it a nine out of ten. 

In our workflow, we initiate the process by fetching data, followed by a preprocessing step to refine the data. We establish pipelines for seamless data flow. The ultimate objective is to transfer this processed data into BigQuery tables, enabling other teams, such as analytics or machine learning, to easily interpret and utilize the information for various purposes, whether it's gaining insights or developing models.

The primary advantages include its speed, especially when dealing with large datasets or big data. It proves exceptionally useful in handling substantial amounts of data efficiently. A notable benefit is the ability to preview data without executing full queries, saving time and allowing for quick insights. This feature eliminates the need to run extensive queries solely for data preview purposes, streamlining the overall workflow.

The interface is what I find particularly valuable. When crafting queries, it offers estimations on data usage, providing a helpful indication of resource consumption. This predictive capability adds an extra layer of convenience, making the querying process more insightful and efficient.

It would be beneficial to integrate additional tools, particularly from a business intelligence perspective. For instance, incorporating machine learning capabilities could enable users to automatically generate SQL queries.

I have been working with it for over a year now.

I find it to be generally high and satisfactory. However, there is a notable issue we've encountered regarding query limitations at the organization level.

It is scalable up to a certain point. There seems to be a restriction on the number of queries one can run, for example, being limited to processing ten terabytes of queries. Exceeding this limit results in an inability to run additional queries, posing a potential challenge. Resolving this limitation could contribute to a smoother user experience. Currently, the user base exceeds two hundred individuals.

We used Google Cloud Storage, IAM, AWS (specifically VPC), and instances from both AWS and Google Cloud Platform. Regarding comparison with other solutions, particularly AWS, there are notable observations. AWS, being introduced earlier, appears to have more extensive features compared to Google Cloud Platform (GCP). AWS enjoys the advantage of having a more established history, resulting in robust support from their team. It offers a more comprehensive platform with a broader range of features, and its pricing structure appears to be more favorable.

The challenging part lies in the initial setup of the project, especially when integrating with project management tools. When establishing a project on the Google Cloud Platform, you need to navigate through various resources.

Setting up the account, whether at an individual or organizational level, involves providing necessary information, including credit card details for billing purposes. Once the account is set up, accessing resources like Cloud Storage or BigQuery becomes straightforward within the Google Cloud Platform.

For those venturing into cloud platforms, especially at an individual level, I would recommend considering AWS. Given its longer establishment in the industry, many companies utilize AWS. Additionally, both AWS and GCP offer free tiers for new users, but AWS extends this benefit to one year, while GCP limits it to three months. At the organizational level, AWS tends to provide more extensive features compared to GCP, making it a preferable choice. Overall, I would rate it eight out of ten.