Try our new research platform with insights from 80,000+ expert users

Apache JMeter vs Katalon Studio comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Sep 11, 2024
 

Categories and Ranking

Apache JMeter
Ranking in API Testing Tools
3rd
Average Rating
7.8
Reviews Sentiment
7.2
Number of Reviews
90
Ranking in other categories
Performance Testing Tools (1st), Load Testing Tools (1st)
Katalon Studio
Ranking in API Testing Tools
5th
Average Rating
7.8
Reviews Sentiment
5.6
Number of Reviews
43
Ranking in other categories
Regression Testing Tools (3rd), Test Automation Tools (3rd)
 

Mindshare comparison

As of November 2024, in the API Testing Tools category, the mindshare of Apache JMeter is 8.0%, down from 16.4% compared to the previous year. The mindshare of Katalon Studio is 15.6%, up from 11.0% compared to the previous year. It is calculated based on PeerSpot user engagement data.
API Testing Tools
 

Featured Reviews

AlexLogan - PeerSpot reviewer
Aug 9, 2022
It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve
JMeter is used by a team of four performance engineers. We monitor tasks on the API level, so all of the requests are hitting individually JMeter has helped us streamline our applications by identifying multiple bottlenecks in our systems and exercising tasks against APIs. JMeter is a free tool…
Anuradha.Kapoor Kapoor - PeerSpot reviewer
May 15, 2024
Functional automation features and the recording functionality saves time but the performance and script execution is slow
I would like to see improvements in how Katalon Studio works with iOS applications, especially with React Native applications. The speed is very slow. It takes time. Katalon Studio also takes my laptop memory, and execution becomes slow for the script. There are parallel execution options. I'm sure their online cloud version must be very fast, but as a service-based company, we are under NDA with our clients and can't run their code on our servers. I would like to see iOS improve and see a more capable recording feature. Katalon Studio has released an AI version, and I have a demo scheduled to see the new features.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"We appreciate that the solution is free to use, as an open-source tool."
"Apache JMeter is well-known and widely used among developers, particularly on popular developer forums. While it may not have the most user-friendly interface, it offers strong support through official manuals and various articles from companies providing load testing services. The tool is free, has a substantial community, and serves as a fundamental choice for testers, especially those new to performance testing. While other tools like K6 may be more developer-oriented, JMeter's affordability and accessibility make it suitable for those without extensive performance testing experience."
"JMeter is basically the art of the entire performance testing process."
"It is very quick and user-friendly."
"Any front-end technologies like Angular, React, or others might access the services. Therefore, you must ensure that using SAP with JMeter works fine. Unless you ensure this, the front-end team may not access the services properly."
"The most valuable features of Apache JMeter are user-friendliness, large resource, and the quality of assistance they provide. Additionally, it is easy to integrate with cloud platforms, such as AWS."
"The solution is scalable."
"Apache JMeter is quite flexible."
"The most valuable feature of Katalon Studio is visual testing. It compares the look and feel of an application which is useful."
"The basic framework and project organization is very good."
"We are now performing automated testing in 15 minutes, which were previously taking a long time when doing it manually."
"Katalon Studio is user-friendly."
"It is easy for a non-technical person to use and adopt."
"Our clients have requested by all types of testing, including mobile, desktop, and API testing and all of those are covered by Katalon. I find that very valuable, very complete."
"Smart Wait and XPath healing are valuable features."
"It supports multiple easy-to-learn languages."
 

Cons

"We're like the solution to be more user-friendly."
"Report generation needs to be improved. It is quite difficult to get to."
"I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this."
"Apache JMeter's UI can be made more colorful."
"We would like more documentation to be provided for the advanced level features that are available in this solution, in order to improve development."
"The only thing is the learning curve. It's high."
"They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available."
"From an improvement perspective, consider a scenario where your application needs to retrieve data from PayPal and store it in an Excel file in an external network."
"Now we are having difficulties in using it and had to limit our automation experts to the features that Katalon allows us to use for free. It was free access and they claimed that it would be free for all time."
"My recently-updated Katalon studio version hangs a lot and is not a stable version."
"It is difficult to identify elements on websites that use Angular or ReactJS because they don't have an option to view source code."
"It seems to lack a few requirements that a typical customer would want. Compared to other solutions, this product is lacking."
"Object Spy is brittle. When I try to capture objects, I have to proceed multiple times before it works."
"It would be helpful if they can integrate a feature for performance testing."
"They need to work on documentation to make it more centralized and easier to find what you are looking for."
"The integration process with Jira should be enhanced and facilitated."
 

Pricing and Cost Advice

"Apache JMeter is a free tool."
"When comparing the price with Load Runner, and if the cost is an issue then JMeter is a better choice"
"I was using the free version of the software."
"This is an open-source solution, so there are no licensing costs involved."
"Since we are using an outsourced solution, it is not paid for by our company."
"We didn't pay licensing fees for Apache JMeter because it's an open-source tool. We only paid for the machines where we installed Apache JMeter modules."
"The tool is open-source."
"Free"
"We are using the free version of Katalon Studio at this point."
"This product is freeware, so there is no charge for using it."
"The cost of Katalon Studio is expensive but it is less than some of the other solutions, such as Micro Focus UFT or SmartBear TestComplete. The cost is the main reason we are using Katalon Studio."
"The tool is affordable."
"Price-wise, Katalon Studio is a costly product. Considering what the solution provides its users with, I can say that price-wise, Katalon Studio is an okay product."
"We are using the free version. We plan on getting a paid version because there is a limitation in terms of the number of users. We want to maximize it."
"Katalon Studio is less expensive than other solutions, such as Micro Focus UFT."
"We were using the free version of this tool."
report
Use our free recommendation engine to learn which API Testing Tools solutions are best for your needs.
815,854 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
19%
Computer Software Company
16%
Manufacturing Company
8%
Government
7%
Educational Organization
32%
Computer Software Company
14%
Financial Services Firm
10%
Manufacturing Company
7%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

How does Postman compare with Apache JMeter?
Postman lets you easily define variables, which then get updated automatically. This is a huge time-saver and makes processes very efficient. We can also export the test cases we create and share t...
How does BlazeMeter compare with Apache JMeter?
Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs. We ...
What do you like most about Apache JMeter?
I appreciate JMeter's simplicity and power for performance testing.
What is your experience regarding pricing and costs for Katalon Studio?
We have a combination of both paid and free licenses. We have five licenses, and we are in the process of buying six more, so six or seven people. The prices are very high. We use it only when our ...
What needs improvement with Katalon Studio?
There could be many improvements, however, I have not noted them down at this moment.
 

Also Known As

JMeter
No data available
 

Overview

 

Sample Customers

AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape
Coca-Cola Tesla Black Board TaTa Consultancy Services Sony
Find out what your peers are saying about Apache JMeter vs. Katalon Studio and other solutions. Updated: October 2024.
815,854 professionals have used our research since 2012.