Apache Spark Reviews

The main feature that we find valuable is that it is very fast. In terms of big data, the main feature is that the data is in so many different nodes. It goes through many data nodes so whenever we use the data, it enables us to parse the data from different data nodes. 

We use big data manager but we cannot use it as conditional data so whenever we're trying to fetch the data, it takes a bit of time. There is some latency in the system and latency in the data caching. The main issue is that we need to design it in a way that data will be available to us very quickly. It takes a long time and the latest data should be available to us much quicked. 

We don't have any problems with stability. 

I'm not the one who would contact their support if we needed it. 

The initial setup is straightforward. 

The advice that I would give to someone considering this solution is that the quality of data has key streaming capabilities like velocity. This means how quickly you are going to refer to the data. These things matter by designing the solution. We need to take these things out. 

I would rate Apache Spark an eight out of ten. 

To make it a ten they should improve the speed. The data storage capacity means we can inject somewhere in the user database in more efficient ways.

I use Apache Spark for any data engineering part. I handle some computation processes where it is necessary to process big data.

Apache Spark's ability to handle both batch and streaming data is the most valuable feature for me. It is beneficial for consuming real-time data. It offers solid real-time processing capability, making it more efficient in managing data analytics. It is beneficial as it allows processing of both batch and streaming data seamlessly.

There is complexity when it comes to understanding the whole ecosystem, especially for beginners. I find it quite complex to understand how a Spark job is initiated, the roles of driver nodes, worker nodes, stages, and tasks. Additionally, clustering may be a bit complex to set up.

I have been using Apache Spark for about two and a half years now.

Clustering may be a bit complex to set up, but it depends on the experience that the person involved has.

I find Apache Spark to be fine and stable.

The scalability of Apache Spark depends on the number of machines being used. By adjusting the worker and driver nodes, scaling can be leveraged.

I haven't tried Apache Spark's official support. I mostly use ChatGPT for assistance.

Neutral

Before Spark, I used solutions like Storm and Flume. In AWS, it is Kinesis. Both Kinesis and Spark have their own ways of managing data injection and compute.

In the public cloud, it comes with built-in services, but for on-premises, I have to spin up my own cluster using my ecosystem.

A single person can handle installation if they are capable enough.

Timing depends on the cluster being used, such as how many compute nodes and the kind of data there is. So, it's not straightforward to specify the percentage of time and money saved.

Apache Spark is open-source, so it doesn't incur any charges.

I used Storm, Flume, and AWS Kinesis.

I rate my overall experience with Apache Spark as eight out of ten. I suggest leveraging AI capabilities to enhance performance or check for anomalies.

Public Cloud

Other