Try our new research platform with insights from 80,000+ expert users

Apache JMeter vs SmartBear TestComplete comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

ROI

Sentiment score
6.7
Apache JMeter, a cost-effective tool, excels in scalability, CICD integration, and ROI, rivaling HP Performance Center, with minimal investment.
Sentiment score
6.7
SmartBear TestComplete automation saves time, enhances client satisfaction, and boosts efficiency, with annual savings of approximately $10,000.
With Apache JMeter, I have gained great statistics for performance and server metrics.
Once set up, only one person is needed to handle all tasks, reducing the requirement for multiple personnel.
 

Customer Service

Sentiment score
6.2
Apache JMeter's community-driven support provides helpful resources, but it's not as immediate or thorough as commercial alternatives.
Sentiment score
6.9
SmartBear TestComplete's customer support is knowledgeable but inconsistent, with delays and unresolved issues needing faster escalation and responses.
The support for Apache JMeter is excellent.
Apache JMeter has strong support through its vast Java-based community on platforms like Stack Overflow.
Apache JMeter relies more on community support.
 

Scalability Issues

Sentiment score
6.6
Apache JMeter is scalable but struggles with high loads, requiring distributed testing and complex configurations for extensive usage.
Sentiment score
7.3
SmartBear TestComplete is scalable and adaptable, with flexible scripting, but may require licensing for wider deployment.
This restricts the number of users and necessitates increasing load agents or distributing the script across multiple machines.
JMeter is highly scalable, easily handling increased loads through the use of multiple servers.
For backend automation and performance testing with web services, web APIs, and queue management systems, I would rate Apache JMeter's scalability as between eight and nine.
 

Stability Issues

Sentiment score
7.2
Apache JMeter is generally stable and reliable but may require tuning for optimal performance under high load conditions.
Sentiment score
6.9
SmartBear TestComplete generally stable, but users report crashes, memory leaks, and HTML5 testing delays in certain scenarios.
JMeter performs exceptionally well, especially in non-GUI mode, which supports high loads efficiently.
Several necessary features still need improvements, specifically in terms of reports and additional functionalities compared to other commercial tools.
 

Room For Improvement

Apache JMeter faces challenges with SSL recording, performance issues, complex scripting, limited support, and inadequate documentation.
SmartBear TestComplete faces challenges in object recognition, integrations, licensing, performance, and support across browsers and mobile devices.
Currently, we need to use multiple separate JMeter instances to simulate reductions in load, which isn't ideal.
Apache JMeter could be significantly enhanced by the integration of BDD frameworks.
Automating report analysis and supporting customized SLAs for script report generation could also enhance functionality.
While using SmartBear TestComplete, we are fine with the current capabilities, however, it would be beneficial to improve some performance aspects, especially the image comparison feature.
 

Setup Cost

Apache JMeter is a cost-effective, open-source tool for smaller projects, with community support; high-scale testing needs additional tools.
SmartBear TestComplete's pricing and licensing receive mixed reviews, seen as both reasonable and costly depending on usage and modules.
Using JMeter helps us avoid additional costs for high-load testing since it is open-source and allows for unlimited virtual users at no extra cost.
Apache JMeter is completely free as it is open-source.
It's a cost-effective solution.
 

Valuable Features

Apache JMeter is a flexible, scalable, open-source tool for comprehensive performance testing with robust community support.
SmartBear TestComplete excels in cross-platform automation, integration, and support for multiple languages, enhancing automated testing efficiency and maintenance.
Despite being open source, it offers features comparable to paid tools.
Monetary benefits with Apache JMeter are notable since it doesn’t require a licensed version.
JMeter facilitates scripting capabilities, which include options for Groovy scripts.
The most valuable feature of SmartBear TestComplete for me is the image comparison functionality.
 

Categories and Ranking

Apache JMeter
Average Rating
7.8
Reviews Sentiment
7.1
Number of Reviews
93
Ranking in other categories
Performance Testing Tools (1st), Load Testing Tools (1st), API Testing Tools (3rd)
SmartBear TestComplete
Average Rating
7.6
Reviews Sentiment
6.8
Number of Reviews
76
Ranking in other categories
Functional Testing Tools (9th), Regression Testing Tools (5th), Test Automation Tools (6th)
 

Mindshare comparison

While both are Quality Assurance solutions, they serve different purposes. Apache JMeter is designed for Performance Testing Tools and holds a mindshare of 24.0%, up 23.9% compared to last year.
SmartBear TestComplete, on the other hand, focuses on Test Automation Tools, holds 5.9% mindshare, down 7.4% since last year.
Performance Testing Tools
Test Automation Tools
 

Featured Reviews

Sreenivasula Mukkamalla - PeerSpot reviewer
Leveraging cost-effective customization with powerful plugins but complexity reduction needed
Apache JMeter offers plugins for reporting and preparing test scenarios. It allows recording to customization, letting you download plugins to connect with databases or external systems. Despite being open source, it offers features comparable to paid tools, and its ability to customize and expand is particularly useful. Additionally, its open-source nature makes it cost-effective.
Prakhar Goel - PeerSpot reviewer
Used for integration automation, user-based automation, and web automation
The solution's most valuable features are the drag-and-drop feature, keyword-driven approach, and reusability of the scripts. The solution has introduced a new feature that helps us identify objects we cannot normally identify. It gives you a fair idea of objects, resolving the object recognition issue. The solution can be used to perform different tests on different machines.
report
Use our free recommendation engine to learn which Performance Testing Tools solutions are best for your needs.
848,989 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
20%
Computer Software Company
16%
Manufacturing Company
7%
Government
7%
Computer Software Company
19%
Manufacturing Company
14%
Financial Services Firm
13%
Government
7%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

How does Postman compare with Apache JMeter?
Postman lets you easily define variables, which then get updated automatically. This is a huge time-saver and makes processes very efficient. We can also export the test cases we create and share t...
How does BlazeMeter compare with Apache JMeter?
Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs. We ...
What do you like most about Apache JMeter?
I appreciate JMeter's simplicity and power for performance testing.
What do you like most about SmartBear TestComplete?
TestComplete has strong reporting capabilities. The reports they generate are really good.
What is your experience regarding pricing and costs for SmartBear TestComplete?
I am not involved in pricing or licensing; our management team handles these aspects.
What needs improvement with SmartBear TestComplete?
While using SmartBear TestComplete, we are fine with the current capabilities, however, it would be beneficial to improve some performance aspects, especially the image comparison feature. Occasion...
 

Also Known As

JMeter
No data available
 

Overview

 

Sample Customers

AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape
Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.
Find out what your peers are saying about Apache, Tricentis, Perforce and others in Performance Testing Tools. Updated: March 2025.
848,989 professionals have used our research since 2012.