Try our new research platform with insights from 80,000+ expert users

SmartBear TestComplete vs TestRail comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Categories and Ranking

SmartBear TestComplete
Average Rating
7.6
Reviews Sentiment
6.8
Number of Reviews
75
Ranking in other categories
Functional Testing Tools (8th), Regression Testing Tools (5th), Test Automation Tools (6th)
TestRail
Average Rating
8.0
Reviews Sentiment
6.8
Number of Reviews
21
Ranking in other categories
Test Management Tools (2nd)
 

Mindshare comparison

While both are Quality Assurance solutions, they serve different purposes. SmartBear TestComplete is designed for Test Automation Tools and holds a mindshare of 6.1%, down 7.6% compared to last year.
TestRail, on the other hand, focuses on Test Management Tools, holds 13.2% mindshare, up 13.2% since last year.
Test Automation Tools
Test Management Tools
 

Featured Reviews

Prakhar Goel - PeerSpot reviewer
Used for integration automation, user-based automation, and web automation
The solution's most valuable features are the drag-and-drop feature, keyword-driven approach, and reusability of the scripts. The solution has introduced a new feature that helps us identify objects we cannot normally identify. It gives you a fair idea of objects, resolving the object recognition issue. The solution can be used to perform different tests on different machines.
StuartBarker - PeerSpot reviewer
A tool that provides effective test management and real-time reporting capabilities
I have faced some issues with the integration between TestRail and Jira where the status of tests is not displayed (in Jira) due to I suspect security settings on the browser. In a large corporate environment, it is not easily changed. The support wasn't particularly helpful. It would be great if I could create custom reports, ideally with a tool designed specifically for that.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"The product is stable for what we are currently using it for, and it is sufficient for us."
"TestComplete is simple, it's a very easy-to-use tool."
"The solution helps improve the stability of our product. It also decreases the work of our manual quality assurance engineers."
"Recording and playback of tests were easier with SmartBear TestComplete...It is a scalable solution."
"You can record your actions and play them back later."
"This company offers end-to-end capabilities for test suite creation and execution. One feature that I particularly appreciate is the tagging system. Tags are highly valuable, as they allow you to assign tags to your test cases. When there's an impact in a specific area, you can search for and run all test cases associated with that tag. I find this functionality very useful."
"SmartBear TestComplete performs some self-healing and has a feature called OCR (optical character recognition)."
"The most valuable feature of this solution is its ability to integrate with Azure DevOps for continuous integration and deployment."
"The most valuable features are the flexibility, ease of use for writing new test cases, the test plans, and the composition."
"The product’s most valuable feature is the UI. The structure of test cases is easy to understand."
"The most valuable features of TestRail by Gurock are the user experience, it's very easy to learn. There is no learning curve needed to work on projects and manage the test cases, it is easy. Exporting and importing are simple."
"You don't need to follow complex procedures to create a test run, test case, etc."
"From a testing perspective, the management is awesome. I am able to do testing and then add the reporting and the evidence. It is fair in terms of the price that you're paying. You get what you're paying for."
"The solution is very stable. We've never had any issues with it."
"I use the product to create test cases and share them with my team and manager."
"Most valuable features are the ease of organizing test cases and a great API for sending results from automated test run results into the database."
 

Cons

"One notable drawback is the absence of native integration with Git."
"The initial setup of SmartBear TestComplete was complex."
"What is currently missing from this solution is better support for mobile testing."
"Stability issues occurred only when connecting to the SourceSafe. Sometimes, after getting the latest version, the tool hangs and it should be reopened in order to recover."
"Right now, the product only supports Windows."
"Right now, when you buy the solution, you need to pay for one solution. You receive one set up and you install it and it's just in that one machine. It would be ideal if they could offer one subscription where you can connect to different machines with a group subscription."
"In the cross-browser domain, it has a few snags with Microsoft Edge and Chrome; although, these problems are not critical."
"This solution could be improved by making it easier to visualize where there is a failure without having to look at it in detail."
"Reporting could be more flexible regarding repeating reports."
"Better prediction of text."
"TestRail should improve its pricing."
"This solution has room for improvement. For example, some particular projects need to adjust access or add additional members and this isn't always possible. Role-based access would improve this."
"I've encountered at some point, some difficulties on the administration side, but I don't remember exactly what they were."
"It's not easy to create a custom report. It's not straightforward. A good improvement would be if there was a way to report and create a custom report without using a plugin or scripting language."
"TestRail's user interface is not great. When you use it for the first time, you will be very uncomfortable and not know how to create test cases. It doesn't have a field for preconditions and post-conditions."
"It would be nice to have a description section when creating the test scenario itself so I can indicate what the configuration should be."
 

Pricing and Cost Advice

"Our ROI is about $10,000 a year."
"The solution's licensing cost has increased because it has moved to some new SLM-based licenses."
"The solution's pricing is too high."
"It comes with a high cost."
"The price of SmartBear TestComplete could be less. The main challenge is when it comes to node-locked. They should use a subscription model, such as a monthly-based subscription or, a quarterly-based subscription. Their floating license is very expensive, and this high price should be reduced or provide, at a minimum, a subscription model."
"The license price for a physical machine is cheap, and for virtual machine, it is very expensive."
"Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day."
"The pricing is pretty reasonable."
"I give the price a five out of ten."
"Pricing for small teams seems correct with respect to competitors."
"Its price is definitely not more. If they introduce automation, they can charge more."
"The price of the solution is based on how many users you have per year. When you grow, it is segmented, For example, 10 to 25, you have a price, and more than 50, or 100, you need to take the enterprise license. I don't think we will reach this point."
"The product is not much expensive."
"My advice to others is to shop around for the best deal. Some options out there are free in cyberspace."
"The product has a reasonable price in terms of the features."
"Use TestRail Cloud (online TR hosted server) and don't worry about maintenance or scalability. It saves a lot of cash and time."
report
Use our free recommendation engine to learn which Test Automation Tools solutions are best for your needs.
838,713 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Computer Software Company
19%
Manufacturing Company
13%
Financial Services Firm
12%
Government
7%
Computer Software Company
21%
Manufacturing Company
10%
Financial Services Firm
10%
Healthcare Company
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about SmartBear TestComplete?
TestComplete has strong reporting capabilities. The reports they generate are really good.
What is your experience regarding pricing and costs for SmartBear TestComplete?
I don't know much about the pricing, however, I think it's cheaper.
What needs improvement with SmartBear TestComplete?
The recording function, when using Python, could be improved, as it does not work well in recording testing.
 

Also Known As

No data available
TestRail by Gurock
 

Overview

 

Sample Customers

Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.
Apple, Microsoft, Boeing, Intel, NASA, Amazon, HP, Samsung
Find out what your peers are saying about SmartBear TestComplete vs. TestRail and other solutions. Updated: April 2023.
838,713 professionals have used our research since 2012.