Try our new research platform with insights from 80,000+ expert users

Apache JMeter vs Katalon Studio comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Sep 11, 2024

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Customer Service

Sentiment score
6.5
Apache JMeter support is community-driven and free, with varying response times; commercial options like BlazeMeter offer dedicated assistance.
Sentiment score
5.8
Katalon Studio's customer service is responsive but varies in quality, with premium options and forums offering better support.
The support for Apache JMeter is excellent.
Apache JMeter relies more on community support.
Katalon's support is not very strong unless you opt for the enterprise version.
Documentation is good.
 

Room For Improvement

Sentiment score
5.5
Apache JMeter faces criticism for its complex GUI, performance issues, limited automation, documentation, and integration challenges.
Sentiment score
4.2
Katalon Studio users seek improved stability, scripting support, integrations, enhanced features, and better pricing to enhance usability and efficiency.
The tool needs improvements related to client-side metrics, integrating with tools like YSlow or HTTP Watch, and enhancing mobile testing capabilities.
Currently, we need to use multiple separate JMeter instances to simulate reductions in load, which isn't ideal.
With BlazeMeter, you can view the results in real-time.
Parallel execution is available, but not with the free version.
I would like Katalon Studio to develop manual test case generation using AI capabilities, as other tools like those using OpenAI are already doing this.
 

Scalability Issues

Sentiment score
6.3
Apache JMeter scales well with distributed testing, enhanced by cloud tools, but faces challenges at high loads and configurations.
Sentiment score
6.8
Katalon Studio offers reasonable scalability and ease of use, though it may limit extensive testing needs for some users.
JMeter is highly scalable, easily handling increased loads through the use of multiple servers.
This restricts the number of users and necessitates increasing load agents or distributing the script across multiple machines.
Without location dependency, it's rated ten out of ten, but with dependency, it can be six out of ten.
 

Setup Cost

Sentiment score
7.8
Apache JMeter is cost-effective and scalable, appealing to enterprises for its flexibility and extensive community support.
Sentiment score
5.5
Katalon Studio is affordable with a free version but advanced features and support incur additional costs, starting at $1,900.
Using JMeter helps us avoid additional costs for high-load testing since it is open-source and allows for unlimited virtual users at no extra cost.
It's a cost-effective solution.
Apache JMeter is completely free as it is open-source.
The approximate cost is around $8,000.
The pricing of Katalon Studio is affordable, making it a sensible option for those looking for an affordable range.
 

Stability Issues

Sentiment score
6.5
Apache JMeter is stable but may require specific configurations for high loads; community support exists for optimal performance.
Sentiment score
6.6
Katalon Studio is generally stable but can be unstable with complex projects, rated 6-9/10 by users.
JMeter performs exceptionally well, especially in non-GUI mode, which supports high loads efficiently.
Several necessary features still need improvements, specifically in terms of reports and additional functionalities compared to other commercial tools.
It needs more stability when test cases are executed.
 

Valuable Features

Sentiment score
8.4
Apache JMeter provides an open-source, scalable solution with extensive plugins, effective reporting, and strong community support for performance testing.
Sentiment score
8.3
Katalon Studio offers versatile, user-friendly testing features with seamless automation, integration, and support for multiple languages and platforms.
Monetary benefits with Apache JMeter are notable since it doesn’t require a licensed version.
Despite being open source, it offers features comparable to paid tools.
JMeter facilitates scripting capabilities, which include options for Groovy scripts.
Katalon Studio has the highest number of integrations compared to other tools at a similar price point, such as Jira, Brokerstack, and Jenkins.
It is good to record and playback in low code.
 

Categories and Ranking

Apache JMeter
Ranking in API Testing Tools
3rd
Average Rating
7.8
Reviews Sentiment
7.1
Number of Reviews
92
Ranking in other categories
Performance Testing Tools (1st), Load Testing Tools (1st)
Katalon Studio
Ranking in API Testing Tools
5th
Average Rating
7.8
Reviews Sentiment
6.6
Number of Reviews
45
Ranking in other categories
Regression Testing Tools (3rd), Test Automation Tools (3rd)
 

Mindshare comparison

As of December 2024, in the API Testing Tools category, the mindshare of Apache JMeter is 7.1%, down from 15.7% compared to the previous year. The mindshare of Katalon Studio is 15.9%, up from 11.3% compared to the previous year. It is calculated based on PeerSpot user engagement data.
API Testing Tools
 

Featured Reviews

AlexLogan - PeerSpot reviewer
It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve
I rate JMeter five out of 10. You need knowledgeable employees to work with the tool because it's difficult to learn. There is a steep learning curve, so you shouldn't expect an entry-level engineer to pick up the tool quickly and become proficient. It takes a considerable amount of time.
Anuradha.Kapoor Kapoor - PeerSpot reviewer
Functional automation features and the recording functionality saves time but the performance and script execution is slow
I would like to see improvements in how Katalon Studio works with iOS applications, especially with React Native applications. The speed is very slow. It takes time. Katalon Studio also takes my laptop memory, and execution becomes slow for the script. There are parallel execution options. I'm sure their online cloud version must be very fast, but as a service-based company, we are under NDA with our clients and can't run their code on our servers. I would like to see iOS improve and see a more capable recording feature. Katalon Studio has released an AI version, and I have a demo scheduled to see the new features.
report
Use our free recommendation engine to learn which API Testing Tools solutions are best for your needs.
823,795 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
19%
Computer Software Company
16%
Manufacturing Company
8%
Government
6%
Educational Organization
32%
Computer Software Company
14%
Financial Services Firm
10%
Manufacturing Company
7%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

How does Postman compare with Apache JMeter?
Postman lets you easily define variables, which then get updated automatically. This is a huge time-saver and makes processes very efficient. We can also export the test cases we create and share t...
How does BlazeMeter compare with Apache JMeter?
Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs. We ...
What do you like most about Apache JMeter?
I appreciate JMeter's simplicity and power for performance testing.
What is your experience regarding pricing and costs for Katalon Studio?
The pricing of Katalon Studio is affordable, making it a sensible option for those looking for an affordable range.
What needs improvement with Katalon Studio?
I would like Katalon Studio to develop manual test case generation using AI capabilities, as other tools like those using OpenAI are already doing this. Also, enhancing its test management aspects ...
 

Also Known As

JMeter
No data available
 

Overview

 

Sample Customers

AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape
Coca-Cola Tesla Black Board TaTa Consultancy Services Sony
Find out what your peers are saying about Apache JMeter vs. Katalon Studio and other solutions. Updated: December 2024.
823,795 professionals have used our research since 2012.