Cloudera Distribution for Hadoop Reviews

We have a solution on top of the Cloudera platform for an electrical distribution company to gather data from smart meters for energy consumption.

This is the only solution that is possible to install on-premise. Cloudera provides a hybrid solution that combines compute on cloud or on-premises. It includes all machine learning algorithms in the Spark machine learning library. 

All functionalities needed for a big data platform and ETL are on the platform, eliminating the need for other tools. It is scalable, ready for vertical scaling, and very powerful, offering numerous functionalities and configurations for generative AI.

We use this solution as a data lake, pre-processing the large amount of data we have for further consumption by relational databases or advanced analytics. We use HDFS and Spark for that purpose and we are using Cloudera Machine Learning, a Jupyter Notebook-like environment with model monitoring opportunities, model catalog, and things like that. We are customers of Cloudera and I'm head of big data and the analytics competency center.

The solution has good features connected to end-to-end security. It's difficult to ensure a safe environment so this is a primary feature and the main reason we use Cloudera is because it's cloud-independent. That's very important because we are considering partially moving our workloads to cloud and not tying ourselves to a given vendor. For instance, if you were to go with Microsoft Azure, it would be impossible to move to another cloud provider if you're not satisfied with their pricing or quality of service.

The tool is used by our company's different customers who have requirements for big data management. When our company's customers want to build a platform for big data management, they choose Cloudera as their tool and as a big data management platform even though there are different options in the market since it is best suited if they consider having an on-premises solution. If a customer wants a cloud-based solution for big data management, then there are other tools in the market that better suit their requirements. For an on-premises big data management platform, Cloudera is the best choice.

The best part of the tool is that it is able to expand horizontally and vertically when its customer wants to grow the business. The tool can be deployed using different container technologies, which makes it very scalable.

We use the product for computing. We mainly use Spark, Hive, HDFS, and Impala.

The product has been instrumental for all computing needs. We have a data warehouse and a data lake. We read from S3 and load it into different databases. We compute all the transformations, logic, and code we write in PySpark or Spark Scala. Spark is very valuable for data processing.

We use the solution for workflow distribution. It's an ETL for real-time and batch-mode processing. It's mainly used for all the stuff, including data warehousing.

The tool's most interesting features are the distributed file system and unstructured data processing capability. Because we have a lot of unstructured data, like XML and social media logs, these features make it more valuable than the usual data warehousing solutions.

Data warehouse solutions mainly use structured, regular, and formatted data, but Cloudera Distribution for Hadoop can handle unstructured data. This is the most interesting part. Also, the huge amount of data can be tuned in HDFS rather than relational databases. Cloudera Distribution for Hadoop can be a promising solution for distributed file systems, real-time processing, batch mode processing, AI, and machine learning use cases.

We are using several security features in the solution. These include Linux's security implementations and its built-in firewall. We also rely on single sign-on and encryption—at rest and in transit—for sensitive data. It has access, ensuring that not everyone can use every service; for example, some users can access Hive, others Impala, and others hBase, depending on their privileges.

We also use LDAP to track who registers or logs into the cluster. Additionally, we use key nodes to manage firewalls between Cloudera Manager or the Cloudera cluster and other data sources.

Cloudera Distribution for Hadoop is used for our data lake and big data solutions.

Big data, from the perspective of the end-user, is almost similar to an open-source solution. However, in the case of Cloudera Distribution, we have support from the Cloudera site, which is good. Additionally, we also engage an external company, which acts as a system integrator, to provide partial involvement in the maintenance and installation of Cloudera Distribution for Hadoop implementations. Nonetheless, our team also works with it, not solely as end users but also when they are developing something on this platform.

There are multiple use cases of Cloudera. It is a big data platform where we collect all the data and connect other sources to get data from multiple sources. Cloudera has a Data Lake.

Data Lake is mature. Their integration across the Hadoop ecosystem and the interoperability across other distributions is mature. The technical skill set for Cloudera is available. They enhance their roadmap quarterly and provide new features to enhance current functionalities and capabilities. They are capitalizing on their product and have a clear roadmap.

We share company data leaks based on cloud data on their clusters.

We had a data warehouse before all the data. We can process a lot more data structures.