Try our new research platform with insights from 80,000+ expert users

Katalon Studio vs Perfecto comparison

 

Comparison Buyer's Guide

Executive Summary
 

Categories and Ranking

Katalon Studio
Ranking in Test Automation Tools
3rd
Average Rating
7.8
Reviews Sentiment
5.6
Number of Reviews
43
Ranking in other categories
Regression Testing Tools (3rd), API Testing Tools (5th)
Perfecto
Ranking in Test Automation Tools
13th
Average Rating
8.4
Number of Reviews
23
Ranking in other categories
Performance Testing Tools (13th), Functional Testing Tools (11th), Mobile App Testing Tools (3rd)
 

Mindshare comparison

As of November 2024, in the Test Automation Tools category, the mindshare of Katalon Studio is 11.6%, up from 11.7% compared to the previous year. The mindshare of Perfecto is 2.7%, down from 3.9% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Test Automation Tools
 

Featured Reviews

Anuradha.Kapoor Kapoor - PeerSpot reviewer
May 15, 2024
Functional automation features and the recording functionality saves time but the performance and script execution is slow
I would like to see improvements in how Katalon Studio works with iOS applications, especially with React Native applications. The speed is very slow. It takes time. Katalon Studio also takes my laptop memory, and execution becomes slow for the script. There are parallel execution options. I'm sure their online cloud version must be very fast, but as a service-based company, we are under NDA with our clients and can't run their code on our servers. I would like to see iOS improve and see a more capable recording feature. Katalon Studio has released an AI version, and I have a demo scheduled to see the new features.
Roland Castelino - PeerSpot reviewer
Aug 4, 2022
Its reporting allows us to have a clear view regarding what tests have been executed
The most valuable would be their Live Stream analysis, where I can see the live analysis of all the executions on a single device or multiple devices as well as track them. The live analysis and reporting would be the single most valuable feature. We leverage Perfecto’s reporting and analytics a lot. From the CI Dashboard, it is mainly the status, which is the past, failure count, and time consumption, e.g., how much time did an average test or script take? Along with that, it provides the historical view compared to the previous result, e.g., am I a pass or fail? Also, the stack trace is very important. Whenever a pass occurs, we don't look beyond that. However, whenever a failure occurs, the stack trace information that it gives us is pretty critical for us when figuring out where failures lie. It gives a summary for the pass/fail count, total test count, the historical view, time consumption for each test as well as the total tests, and the stack rate of the failure. Perfecto's analytics are very important since we use them on a daily basis. We run our executions daily. After every execution, we pull information from the Perfecto reporting system and share that with our stakeholders. Having this information accurately reported is pretty important for us, so everybody is aware of the current status of the product. That way, we can evaluate the health of the product or environment against that which has been executed. Therefore, it helps make those real-time decisions and highlights the impact to the business. I found Perfecto to be pretty easy to use while executing against cross-platforms. The main reason is because the same script or test automation where we execute on multiple platforms has minimal changes that I need to do. Also, it is easy for me to set up an execution on one platform, then on another platform, either in parallel or one after the other. Parallel opportunities save me time. Once the execution has been completed across these different configurations, I can always check and compare, e.g., what are the differences and consistencies? We utilize Perfecto’s cloud-based lab to test across devices, browsers, and OSs. I use it occasionally for manual testing. Though, there are other team members who use it more frequently than I do. I use it mainly for executing my automated tests. We have the Perfecto lab, cloud devices, and machines. I can program my test to execute against any of those devices, which gives me more confidence in my product. I can compare and see how my product or application functionally behaves across these different devices and from a UI point of view, which helps me a lot. The device lab is extremely important to our testing operations. We rely on having multiple devices up and running all the time. Whenever we kick off an execution, there are multiple reasons why executions may get triggered: * CodeCommit * A scheduled job. * Might be on-demand by any stakeholder. We need the lab to be available, as we need devices up and running for executions to take place. Also, the devices help since they allow us to have parallel execution, and not just wait for a sequential device to become free and available. Therefore, volume is definitely key. It also gives us an opportunity to compare execution across platforms in that space. It is extremely important to you that the lab provides same-day access to new devices since we analyze that data every single day after execution. Perfecto provides their own framework called Quantum Framework. That is one option. The other option is, if I want to have my own framework, I can have a Java-based Maven project, take a Selenium library, AppiumLibrary, and REST Assured library, and utilize the open-source framework. It is easy for us to connect to Perfecto, no matter what framework we use, as long as it has these core libraries in it. I can design and structure it any way that I want. The execution will happen in Perfecto no matter what since they have support for these tools or libraries. It is pretty neat that way. We are not dependent on using just one particular framework to use Perfecto. While there are still some framework limitations, there is the opportunity to use multiple, different open-source frameworks, then pass the execution to Perfecto. We can use most frameworks, then design and craft it any way that we want, then just pass the execution to Perfecto.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"User-friendly and a good solution."
"The product provides ease of automation for the cloud."
"Our clients have requested by all types of testing, including mobile, desktop, and API testing and all of those are covered by Katalon. I find that very valuable, very complete."
"This solution is very user-friendly to learn and implement, and less technical knowledge is required to handle automation."
"Katalon Studio is user-friendly."
"Whenever I speak to the support team of Katalon Studio, I learn that some new features will be included in the product. It is a scalable product."
"I rate Katalon Studio's scalability a four out of five."
"It has been good so far for API testing on Mac. It is not that hard to learn and use. There is so much support out there, so if anyone wants to start using it, there is enough help."
"Their team is really great to work with. They're very flexible, and they always show care. They prioritize our work, our company, and our working relationship. I appreciate the ad hoc sessions that they schedule to provide help with troubleshooting, provide the information that we're looking for, or do a demo of a new feature that they have. They're always willing and very quick to get that scheduled for us. I appreciate that a lot."
"The reporting feature is really tough to find in some of the other products that are competitors. Having your CITB type dashboard, where we can see the test results and see recordings of each test that passed or failed, is probably one of the distinguishing aspects of Perfecto."
"The quality of our software has improved since we implemented this solution."
"The automation piece is the most valuable feature. Every time we had a new version of either OS or an application, we found that being able to automate the testing across different devices is very valuable."
"In terms of Perfecto's ability to perform cross-platform testing, I would rate it a ten out of ten."
"In terms of cross-platform testing, they offer all of it, every device available in the market. It covers real scenarios that mimic production so that we don't miss out on any devices that our clients might be using to run the applications we develop. It's been great and very helpful."
"It saves on the cost and effort of having to maintain our own virtual testing environment. Even our onshore team is not in the city that we work in, so that helps a lot. Even if we didn't invest a lot in getting multiple devices, having to share those devices would become a hassle."
"It creates a faster production cycle and is quick to market. Things get deployed earlier because the testing happens on time. We can do a lot of panelization, so a lot of test phases can happen in a panel. People don't have to wait for a device to come to them. They can access multiple devices at the same time and do testing at the same time."
 

Cons

"Its stability needs improvement."
"I would like to see improvements in how Katalon Studio works with iOS applications, especially with React Native applications."
"It should start working on Silverlight automation."
"During parallel execution, some reports are randomly switched."
"It would be helpful to have an automatic save feature."
"My recently-updated Katalon studio version hangs a lot and is not a stable version."
"What it lacks is the Selenium Grid capabilities."
"There is not much support offered for the free version, which is something that could be improved."
"When using devices on the cloud, it lags quite a bit at times. I know that these are real devices that are being projected on our laptop screens and monitors, but if the speed could be improved, that would be good."
"One improvement would be speed of execution. If it is an iOS native app, we have noticed that the speed is a bit slower. Perfecto might need to make some improvements in this area."
"I'm hoping they can support on-premises instances. We have been working on a JIRA integration with Perfecto—and I'm extremely impressed that they have that—but at this time they're not supporting onsite JIRA instances, which is what we have."
"We don't use Perforce's BlazeMeter with Perfecto. From my perspective, it's not really relevant."
"We have had some issues with performance, which is something that should be improved."
"Going by the dashboard or analytics capabilities that Perfecto or Perforce is looking to offer in its roadmap, it will certainly help if they also cater to executing and enabling decision-making, rather than just focusing on standard testing metrics such as execution, efficiency, and defect rate. These are good metrics, but they don't necessarily enable decision-making for SLTs. Any improvements in the dashboards and reporting tools should focus on metrics or SLAs that can help with decision-making."
"If we could run an accessibility test in Perfecto against builds, it would help us a lot. Currently, that's a very manual process for us. We haven't found a tool that can do accessibility scans for iOS and doesn't depend on engineering effort. Having a feature related to that would be really awesome for us."
"It does well for mobile testing, but when it comes to the web aspect, it is lagging a little bit in terms of execution."
 

Pricing and Cost Advice

"In terms of pricing, Katalon, Selenium, and Cypress all represent a similar level of investment. The software for these tools are either true open source or almost open source (because they charge a small fee), so pricing is a neutral point for me."
"Katalon Studio is a free tool."
"The solution is more affordable than other vendors."
"This product is freeware, so there is no charge for using it."
"Price-wise, Katalon Studio is a costly product. Considering what the solution provides its users with, I can say that price-wise, Katalon Studio is an okay product."
"We are using the free version. We plan on getting a paid version because there is a limitation in terms of the number of users. We want to maximize it."
"The tool's pricing is cheap compared to other alternatives. I would rate the tool's pricing a three out of ten."
"The tool's licensing costs are yearly."
"Perfecto has definitely saved us on the costs and efforts of having to maintain our own virtual test environment. We lost about 20 devices in the past to maintenance and audit. That was a massive loss for us, as a company, because we were giving devices to someone, but don't know whether we would get it back or not. Having those virtual labs, we don't need to worry about these kinds of things. We are easily saving $5,000 to $10,000 a month on device costing."
"I am not sure about its pricing, but from our perspective, licensing has been easy. Anytime I have new users or requests for users that want to get added, it's a very simple process. I just give the architectural owner of the product the name and email address, and they're able to easily add a new user. We don't have any issues in regards to getting licenses, but I don't have any insights into pricing."
"Pricing-wise, it is fine. It is not as expensive as what we used to have in the past from HP, IBM, and others. It is decently priced."
"This is an expensive solution compared to others, by 30% to 40%."
"Pricing is an area where Perfecto can do a little better. When we obtain additional licenses, we enter into negotiations with them."
"Although Perfecto is a good product for us to use, it is a bit expensive. It takes management a bit of work to find the appropriate funding for us to keep Perfecto. I imagine there could be some way to make it more accessible."
"Perfecto is about 30-40% cheaper than Device Anywhere. That was a big reason why we switched. Perfecto also solves some of the issues that we had with Device Anywhere. We have grown by 100% since we started to use Perfecto, and now we have devices roaming. When we look at the competition, we would still stick with Perfecto."
"It's definitely on the higher end of prices for this type of service."
report
Use our free recommendation engine to learn which Test Automation Tools solutions are best for your needs.
814,649 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Educational Organization
32%
Computer Software Company
14%
Financial Services Firm
10%
Manufacturing Company
7%
Financial Services Firm
20%
Computer Software Company
17%
Manufacturing Company
9%
Recreational Facilities/Services Company
7%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What is your experience regarding pricing and costs for Katalon Studio?
We have a combination of both paid and free licenses. We have five licenses, and we are in the process of buying six more, so six or seven people. The prices are very high. We use it only when our ...
What needs improvement with Katalon Studio?
There could be many improvements, however, I have not noted them down at this moment.
What do you like most about Perfecto?
Mobile testing is the most valuable feature as it has reduced dependency on physical devices. We are located offshore and we don't have the physical devices, and shipping physical devices after eve...
What is your experience regarding pricing and costs for Perfecto?
Perfecto's price is excellent compared to other products with similar features. It was the lowest of the three we evaluated. We also established a partnership with Perfecto, so they provide discoun...
What needs improvement with Perfecto?
It is slow compared to physical device testing. The interactive speed could be improved. And sometimes we have issues with our app not working properly with Perfecto. Even though it is working perf...
 

Comparisons

 

Also Known As

No data available
Perfecto Mobile, Perfecto Web
 

Overview

 

Sample Customers

Coca-Cola Tesla Black Board TaTa Consultancy Services Sony
Virgin Media, Paychex, Rabobank, R+V, Discover
Find out what your peers are saying about Katalon Studio vs. Perfecto and other solutions. Updated: October 2024.
814,649 professionals have used our research since 2012.