Try our new research platform with insights from 80,000+ expert users

SmartBear TestComplete vs TestRail comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Categories and Ranking

SmartBear TestComplete
Average Rating
7.6
Reviews Sentiment
6.8
Number of Reviews
76
Ranking in other categories
Functional Testing Tools (9th), Regression Testing Tools (5th), Test Automation Tools (6th)
TestRail
Average Rating
8.0
Reviews Sentiment
7.3
Number of Reviews
22
Ranking in other categories
Test Management Tools (2nd)
 

Mindshare comparison

While both are Quality Assurance solutions, they serve different purposes. SmartBear TestComplete is designed for Test Automation Tools and holds a mindshare of 5.9%, down 7.4% compared to last year.
TestRail, on the other hand, focuses on Test Management Tools, holds 12.2% mindshare, down 13.4% since last year.
Test Automation Tools
Test Management Tools
 

Featured Reviews

Prakhar Goel - PeerSpot reviewer
Used for integration automation, user-based automation, and web automation
The solution's most valuable features are the drag-and-drop feature, keyword-driven approach, and reusability of the scripts. The solution has introduced a new feature that helps us identify objects we cannot normally identify. It gives you a fair idea of objects, resolving the object recognition issue. The solution can be used to perform different tests on different machines.
StuartBarker - PeerSpot reviewer
A tool that provides effective test management and real-time reporting capabilities
I have faced some issues with the integration between TestRail and Jira where the status of tests is not displayed (in Jira) due to I suspect security settings on the browser. In a large corporate environment, it is not easily changed. The support wasn't particularly helpful. It would be great if I could create custom reports, ideally with a tool designed specifically for that.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"When compared to other tools, it is very simple."
"The ability to run a whole suite of tests automatically (which we did overnight)."
"Runs in different remote machines. We have multiple versions of the software being tested."
"It's cross platform automation capabilities specially ranging across web, UNIX (via putty), and other systems."
"It is very easy to maintain tests with this tool. It covers all necessary items in the test plan. The most painful item in testing is maintenance. When changes occur, the tests should be maintained."
"The solution's most valuable features are the drag-and-drop feature, keyword-driven approach, and reusability of the scripts."
"The initial setup is pretty easy and it's quick to deploy."
"In TestComplete, I saw a conformed package of a tool that kept everybody in consistency. The team was able to regenerate further tests without having to manipulate more code because the record feature is great."
"The integration with Jira and the ability for extra configurations are the most valuable features."
"You don't need to follow complex procedures to create a test run, test case, etc."
"The most valuable features of TestRail by Gurock are the user experience, it's very easy to learn. There is no learning curve needed to work on projects and manage the test cases, it is easy. Exporting and importing are simple."
"The ability to time test runs gives the tester the ability to compare calculated times to actual times it takes for a test case to run."
"Integration with Confluence and JIRA."
"The biggest advantage of using TestRail is its simplicity and intuitive interface."
"The API to support integration of the homemade automated testing tool."
"From a testing perspective, the management is awesome. I am able to do testing and then add the reporting and the evidence. It is fair in terms of the price that you're paying. You get what you're paying for."
 

Cons

"In SmartBear TestComplete the integration with Jenkins could be easier. Additionally, some of the controls could have better customization options. For example, if a grid is used and it contains multiple controls within it, it can be a checkbox, radio button, or any kind of control, the way the Object Spy is operating currently there is a lot of room for improvement."
"If that engine could better identify more XPaths automatically and make the process more flexible, that would be better."
"TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services."
"It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing)."
"Product is not stable enough and it crashes often."
"There could be API interfaces with this tool."
"The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS."
"The solution’s customer support should be improved."
"I have faced some issues with the integration between TestRail and Jira, which haven't been permanently resolved yet."
"I've encountered at some point, some difficulties on the administration side, but I don't remember exactly what they were."
"It would be nice if they would add an export to Word."
"It would be useful if it had its own issue management system. At the moment, it's purely a test management tool and you have to link to a defect management tool, like JIRA. It would be useful if there was an option to use its own defect management tool so that it's integrated and not two separate tools."
"It would be nice to have a description section when creating the test scenario itself so I can indicate what the configuration should be."
"There are a number of improvements that have been requested. While I don't have a list of these requests available, many can be found on Gurock's forum."
"The platform needs improvement regarding performance and creating links."
"Customer support could be enhanced. All the support is provided via chat and emails. Sometimes, you really need to speak to a person, and arranging such calls is not easy."
 

Pricing and Cost Advice

"It comes with a high cost."
"The price of SmartBear TestComplete could be less. The main challenge is when it comes to node-locked. They should use a subscription model, such as a monthly-based subscription or, a quarterly-based subscription. Their floating license is very expensive, and this high price should be reduced or provide, at a minimum, a subscription model."
"My advice so far, is that while it’s not quite as powerful and easy to use as UFT, its price tag more than makes up for it."
"The product is becoming more and more expensive."
"The solution's licensing cost has increased because it has moved to some new SLM-based licenses."
"This is a pay-per-use service that is not expensive, and cost-efficient if you have a small team."
"It costs a few hundred per year, but I am not sure. It is not at all expensive as compared to other tools."
"It is approximately $6,000 a year."
"Its price is definitely not more. If they introduce automation, they can charge more."
"My advice to others is to shop around for the best deal. Some options out there are free in cyberspace."
"The product has a reasonable price in terms of the features."
"The price of the solution is based on how many users you have per year. When you grow, it is segmented, For example, 10 to 25, you have a price, and more than 50, or 100, you need to take the enterprise license. I don't think we will reach this point."
"The solution is quite reasonably priced for what it offers and offers a monthly subscription model."
"Negotiate the best deal you can."
"I give the price a five out of ten."
"Pricing for small teams seems correct with respect to competitors."
report
Use our free recommendation engine to learn which Test Automation Tools solutions are best for your needs.
844,944 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Computer Software Company
20%
Manufacturing Company
14%
Financial Services Firm
12%
Government
7%
Computer Software Company
20%
Financial Services Firm
10%
Manufacturing Company
10%
Insurance Company
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about SmartBear TestComplete?
TestComplete has strong reporting capabilities. The reports they generate are really good.
What is your experience regarding pricing and costs for SmartBear TestComplete?
I don't know much about the pricing, however, I think it's cheaper.
What needs improvement with SmartBear TestComplete?
The recording function, when using Python, could be improved, as it does not work well in recording testing.
 

Also Known As

No data available
TestRail by Gurock
 

Overview

 

Sample Customers

Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.
Apple, Microsoft, Boeing, Intel, NASA, Amazon, HP, Samsung
Find out what your peers are saying about SmartBear TestComplete vs. TestRail and other solutions. Updated: April 2023.
844,944 professionals have used our research since 2012.