Apache Spark Reviews

We primarily use the solution to integrate very large data sets from another environment, such as our SQL environment, and draw purposeful data before checking it. We also use the solution for streaming very very large servers. 

It is a very fast solution. It's very easy to use. There are many RPis with many languages like Scala, Java, R, and Python. The greatest advantage of Spark is that we can initiate many kinds of analytics including SQL analytics, graphics analytics, etc. 

The solution needs to optimize shuffling between workers.

I've been using the solution for four or five years.

The solution is very stable.

The solution is scalable. My understanding is version 3.0 has renewed scaling capabilities and will be able to do so automatically.

Apache is an open-source platform so there is no technical support.

We use both on-premises and public and private cloud deployment models. We're partners with Databricks.

I'm a consultant. Our company works for large enterprises such as banks and energy companies. 17 of our workers use Apache Spark.

With the cloud, there are many companies that integrate Spark. Most projects in big data around the world use Spark, indirectly or directly. 

I'd rate the solution eight out of ten.

Used for building big data platforms for processing huge volumes of data. Additionally, streaming data is critical.

It provides a scalable machine learning library so that we can train and predict user behavior for promotion purposes.

Machine learning, real time streaming, and data processing are fantastic, as well as the resilient or fault tolerant feature.

I would suggest for it to support more programming languages, and also provide an internal scheduler to schedule spark jobs with monitoring capability.

The most valuable feature is the Fault Tolerance and easy binding with other processes like Machine Learning, graph analytics. The community is growing and hence executing ML in a distributed fashion is quite good.

Previously we were using Hadoop MapReduce to reduce the Google Ngrams (3TB), which took us approximately five days on our cluster. After using Spark, we were able to accomplish this task within hours.

This product is already improving as the community is developing it rapidly. More ML based algorithms should be added to it, to make it algorithmic-rich for developers.

Two and a half years.

No, I did not encounter any problems with the stability. It is also quite backwards compatible.

No I did not as of now, it is quite scalable. Using simple scripts you can add as many workers as you want.

This is a very good product for the big data analytics and integrates well with other parts like Machine Learning and graph analytics.

It allows the loading and investigation of very lard data sets, has MLlib for machine learning, Spark streaming, and both the new and old dataframe API.

We're able to perform data discovery on large datasets without too much difficulty.

It needs better documentation as well as examples for all the Spark libraries. That would be very helpful in maximizing its capabilities and results.

I've used it for over nine months now.

I haven't encountered any issues with deployment.

There have been no stability issues.

I haven't had any scalability issues. It scales better than Python and R.

I haven't had to use customer service.

I haven't had to use technical support.

I previously used Python and R, but neither of these scaled particularly well.

The initial setup was complex. It was not easy getting the correct version and dependencies set up.

I implemented it in-house on my own!

It's open-source, so ROI is inapplicable.

Learn Scala as this will greatly reduce the pain in starting off with Spark.

\Spark Streaming, Spark SQL and MLib in that order.

We have been using Spark to do a lot of batch and stream processing of inbound data from Apache Kafka. Scaling Spark on YARN is still an issue but we are getting acceptable performance.

Like I said scalability is still an issue, also stability. Spark on Yarn still doesn't seem to have programming submission api, so have to rely on spark-submit script to run jobs on YARN. Scala vs Java API have performance differences which will require sometimes to code in Scala.

Have Scala developers at hand. Base Java competency will not be enough during optimization rounds.

I am using Apache Spark for the data transition from databases. We have customers who have one database as a data lake.

The most valuable feature of Apache Spark is its ease of use.

Apache Spark can improve the use case scenarios from the website. There is not any information on how you can use the solution across the relational databases toward multiple databases.  

I have been using Apache Spark for approximately 18 months.

Apache Spark is stable.

We are using Apache Spark across multiple nodes and it is scalable.

We have approximately five people using this solution.

The technical support from Apache Spark is very good.

I rate Apache Spark an eight out of ten.

We primarily use the solution for security analytics.

The scalability has been the most valuable aspect of the solution.

The management tools could use improvement. Some of the debugging tools need some work as well. They need to be more descriptive. 

The 2.3 version is quite stable. All of our customers use it, there are around 100,000+ users, and it runs 24/7.

The scalability is very good.

You actually buy Cloudera along with it. You don't really get any support, except you need support.

In previous companies, we used MySQL platform and solutions like ArcSight and Splunk. We switched for scalability. MySQL wasn't going to scale, and we don't use Splunk at this company.

The initial setup was complex. It is a complex tool. It's a lot to do with how you will use it. There is a lot to set up. They need to put a lot of scripts to it. There's nearly 60 to set up. When you set up the cloud, it takes about a day to set up. If you set it up on-premise, you know, on hardware, it only takes about a week.

I would rate this solution eight out of 10. 

The main feature that we find valuable is that it is very fast. In terms of big data, the main feature is that the data is in so many different nodes. It goes through many data nodes so whenever we use the data, it enables us to parse the data from different data nodes. 

We use big data manager but we cannot use it as conditional data so whenever we're trying to fetch the data, it takes a bit of time. There is some latency in the system and latency in the data caching. The main issue is that we need to design it in a way that data will be available to us very quickly. It takes a long time and the latest data should be available to us much quicked. 

We don't have any problems with stability. 

I'm not the one who would contact their support if we needed it. 

The initial setup is straightforward. 

The advice that I would give to someone considering this solution is that the quality of data has key streaming capabilities like velocity. This means how quickly you are going to refer to the data. These things matter by designing the solution. We need to take these things out. 

I would rate Apache Spark an eight out of ten. 

To make it a ten they should improve the speed. The data storage capacity means we can inject somewhere in the user database in more efficient ways.