Try our new research platform with insights from 80,000+ expert users

Apache JMeter vs Perfecto comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Customer Service

Sentiment score
6.5
Apache JMeter lacks official support but benefits from a strong, helpful community offering prompt assistance and extensive online resources.
No sentiment score available
The support for Apache JMeter is excellent.
 

Room For Improvement

Sentiment score
5.3
Apache JMeter faces challenges with SSL, scalability, UI, performance, integration, complexity, documentation, user-friendliness, plugins, and scripting.
No sentiment score available
With BlazeMeter, you can view the results in real-time.
Currently, we need to use multiple separate JMeter instances to simulate reductions in load, which isn't ideal.
 

Scalability Issues

Sentiment score
6.6
Apache JMeter is scalable and customizable but faces challenges with memory, setup, and performance compared to paid tools.
No sentiment score available
JMeter is highly scalable, easily handling increased loads through the use of multiple servers.
 

Setup Cost

Sentiment score
7.9
Apache JMeter is a free, open-source tool favored for its cost-effectiveness despite potential additional integration or infrastructure expenses.
No sentiment score available
Using JMeter helps us avoid additional costs for high-load testing since it is open-source and allows for unlimited virtual users at no extra cost.
Monetary benefits with Apache JMeter are notable as it doesn't require a licensed version, whereas BlazeMeter involves costs.
 

Stability Issues

Sentiment score
6.7
Apache JMeter is stable but may require optimizations for high loads; expert users find it more reliable.
No sentiment score available
JMeter performs exceptionally well, especially in non-GUI mode, which supports high loads efficiently.
 

Valuable Features

Sentiment score
8.5
Apache JMeter is a popular, affordable, and versatile testing tool appreciated for its usability, extensibility, and comprehensive feature set.
No sentiment score available
It's useful for both the person conducting the test and the higher management, like project managers or senior executives, who may not know about the test.
JMeter's open-source nature allows us to create scripts easily, add any number of users during load tests, and handle large volumes of data without additional cost.
 

Categories and Ranking

Apache JMeter
Ranking in Performance Testing Tools
1st
Average Rating
7.8
Reviews Sentiment
7.2
Number of Reviews
90
Ranking in other categories
Load Testing Tools (1st), API Testing Tools (3rd)
Perfecto
Ranking in Performance Testing Tools
13th
Average Rating
8.4
Reviews Sentiment
7.9
Number of Reviews
23
Ranking in other categories
Functional Testing Tools (11th), Mobile App Testing Tools (3rd), Test Automation Tools (13th)
 

Mindshare comparison

As of November 2024, in the Performance Testing Tools category, the mindshare of Apache JMeter is 25.0%, up from 22.0% compared to the previous year. The mindshare of Perfecto is 0.9%, up from 0.6% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Performance Testing Tools
 

Featured Reviews

AlexLogan - PeerSpot reviewer
It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve
I rate JMeter five out of 10. You need knowledgeable employees to work with the tool because it's difficult to learn. There is a steep learning curve, so you shouldn't expect an entry-level engineer to pick up the tool quickly and become proficient. It takes a considerable amount of time.
Roland Castelino - PeerSpot reviewer
Its reporting allows us to have a clear view regarding what tests have been executed
The most valuable would be their Live Stream analysis, where I can see the live analysis of all the executions on a single device or multiple devices as well as track them. The live analysis and reporting would be the single most valuable feature. We leverage Perfecto’s reporting and analytics a lot. From the CI Dashboard, it is mainly the status, which is the past, failure count, and time consumption, e.g., how much time did an average test or script take? Along with that, it provides the historical view compared to the previous result, e.g., am I a pass or fail? Also, the stack trace is very important. Whenever a pass occurs, we don't look beyond that. However, whenever a failure occurs, the stack trace information that it gives us is pretty critical for us when figuring out where failures lie. It gives a summary for the pass/fail count, total test count, the historical view, time consumption for each test as well as the total tests, and the stack rate of the failure. Perfecto's analytics are very important since we use them on a daily basis. We run our executions daily. After every execution, we pull information from the Perfecto reporting system and share that with our stakeholders. Having this information accurately reported is pretty important for us, so everybody is aware of the current status of the product. That way, we can evaluate the health of the product or environment against that which has been executed. Therefore, it helps make those real-time decisions and highlights the impact to the business. I found Perfecto to be pretty easy to use while executing against cross-platforms. The main reason is because the same script or test automation where we execute on multiple platforms has minimal changes that I need to do. Also, it is easy for me to set up an execution on one platform, then on another platform, either in parallel or one after the other. Parallel opportunities save me time. Once the execution has been completed across these different configurations, I can always check and compare, e.g., what are the differences and consistencies? We utilize Perfecto’s cloud-based lab to test across devices, browsers, and OSs. I use it occasionally for manual testing. Though, there are other team members who use it more frequently than I do. I use it mainly for executing my automated tests. We have the Perfecto lab, cloud devices, and machines. I can program my test to execute against any of those devices, which gives me more confidence in my product. I can compare and see how my product or application functionally behaves across these different devices and from a UI point of view, which helps me a lot. The device lab is extremely important to our testing operations. We rely on having multiple devices up and running all the time. Whenever we kick off an execution, there are multiple reasons why executions may get triggered: * CodeCommit * A scheduled job. * Might be on-demand by any stakeholder. We need the lab to be available, as we need devices up and running for executions to take place. Also, the devices help since they allow us to have parallel execution, and not just wait for a sequential device to become free and available. Therefore, volume is definitely key. It also gives us an opportunity to compare execution across platforms in that space. It is extremely important to you that the lab provides same-day access to new devices since we analyze that data every single day after execution. Perfecto provides their own framework called Quantum Framework. That is one option. The other option is, if I want to have my own framework, I can have a Java-based Maven project, take a Selenium library, AppiumLibrary, and REST Assured library, and utilize the open-source framework. It is easy for us to connect to Perfecto, no matter what framework we use, as long as it has these core libraries in it. I can design and structure it any way that I want. The execution will happen in Perfecto no matter what since they have support for these tools or libraries. It is pretty neat that way. We are not dependent on using just one particular framework to use Perfecto. While there are still some framework limitations, there is the opportunity to use multiple, different open-source frameworks, then pass the execution to Perfecto. We can use most frameworks, then design and craft it any way that we want, then just pass the execution to Perfecto.
report
Use our free recommendation engine to learn which Performance Testing Tools solutions are best for your needs.
816,406 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
19%
Computer Software Company
16%
Manufacturing Company
8%
Government
7%
Financial Services Firm
20%
Computer Software Company
17%
Manufacturing Company
9%
Recreational Facilities/Services Company
7%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

How does Postman compare with Apache JMeter?
Postman lets you easily define variables, which then get updated automatically. This is a huge time-saver and makes processes very efficient. We can also export the test cases we create and share t...
How does BlazeMeter compare with Apache JMeter?
Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs. We ...
What do you like most about Apache JMeter?
I appreciate JMeter's simplicity and power for performance testing.
What do you like most about Perfecto?
Mobile testing is the most valuable feature as it has reduced dependency on physical devices. We are located offshore and we don't have the physical devices, and shipping physical devices after eve...
What needs improvement with Perfecto?
It is slow compared to physical device testing. The interactive speed could be improved. And sometimes we have issues with our app not working properly with Perfecto. Even though it is working perf...
What is your primary use case for Perfecto?
We use it for manual mobile testing and a little bit of web testing.
 

Comparisons

 

Also Known As

JMeter
Perfecto Mobile, Perfecto Web
 

Learn More

 

Overview

 

Sample Customers

AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape
Virgin Media, Paychex, Rabobank, R+V, Discover
Find out what your peers are saying about Apache JMeter vs. Perfecto and other solutions. Updated: October 2024.
816,406 professionals have used our research since 2012.