Try our new research platform with insights from 80,000+ expert users

SmartBear TestComplete vs TestRail comparison

 

Comparison Buyer's Guide

Executive Summary
 

Categories and Ranking

SmartBear TestComplete
Average Rating
7.6
Number of Reviews
74
Ranking in other categories
Functional Testing Tools (7th), Regression Testing Tools (5th), Test Automation Tools (6th)
TestRail
Average Rating
8.0
Number of Reviews
21
Ranking in other categories
Test Management Tools (2nd)
 

Mindshare comparison

While both are Quality Assurance solutions, they serve different purposes. SmartBear TestComplete is designed for Test Automation Tools and holds a mindshare of 6.5%, down 7.9% compared to last year.
TestRail, on the other hand, focuses on Test Management Tools, holds 13.1% mindshare, down 13.3% since last year.
Test Automation Tools
Test Management Tools
 

Featured Reviews

Prakhar Goel - PeerSpot reviewer
Jun 25, 2024
Used for integration automation, user-based automation, and web automation
We have applications related to power plants, and we use the solution to do integration automation, user-based automation, and web automation The solution's most valuable features are the drag-and-drop feature, keyword-driven approach, and reusability of the scripts. The solution has introduced a…
StuartBarker - PeerSpot reviewer
Sep 4, 2023
A tool that provides effective test management and real-time reporting capabilities
We use the solution for test management during projects The solution helps us with the general management of projects. It provides us with visibility on progress and quality. The most valuable feature of the solution is that it's an excellent repository for TestRail suite. TestRail provides…

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"The product has many features."
"You can record your actions and play them back later."
"It is very easy to maintain tests with this tool. It covers all necessary items in the test plan. The most painful item in testing is maintenance. When changes occur, the tests should be maintained."
"The ability to run a whole suite of tests automatically (which we did overnight)."
"TestComplete is simple, it's a very easy-to-use tool."
"The database checkpoints detect problems which are difficult for a human resource to find."
"SmartBear TestComplete performs some self-healing and has a feature called OCR (optical character recognition)."
"Selenium integration."
"The most valuable features of TestRail by Gurock are the user experience, it's very easy to learn. There is no learning curve needed to work on projects and manage the test cases, it is easy. Exporting and importing are simple."
"The most valuable features are the flexibility, ease of use for writing new test cases, the test plans, and the composition."
"This is a user friendly solution."
"Reliable and stable. It is important that TestRail be up and running 24/7 as we have users around the world using it."
"You don't need to follow complex procedures to create a test run, test case, etc."
"The product’s most valuable feature is the UI. The structure of test cases is easy to understand."
"Most valuable features are the ease of organizing test cases and a great API for sending results from automated test run results into the database."
"The API to support integration of the homemade automated testing tool."
 

Cons

"Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work."
"If that engine could better identify more XPaths automatically and make the process more flexible, that would be better."
"The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS."
"The solution needs more training manuals or some form of online forum for learning. It needs more documentation."
"Stability issues occurred only when connecting to the SourceSafe. Sometimes, after getting the latest version, the tool hangs and it should be reopened in order to recover."
"In the cross-browser domain, it has a few snags with Microsoft Edge and Chrome; although, these problems are not critical."
"During the distribution of our regression test cases, the control IDs are not always recognized correctly."
"We were testing handheld barcode scanners running WindowsCE with many menus of warehouse functions, and our biggest problem was the timing between input and responses."
"The product is not focused on synthetic data creation. I would also like to see more integrations with other platforms."
"TestRail's user interface is not great. When you use it for the first time, you will be very uncomfortable and not know how to create test cases. It doesn't have a field for preconditions and post-conditions."
"TestRail should improve its pricing."
"It would be useful if it had its own issue management system. At the moment, it's purely a test management tool and you have to link to a defect management tool, like JIRA. It would be useful if there was an option to use its own defect management tool so that it's integrated and not two separate tools."
"I do see room for lots of improvement in it. In terms of usability, duplication with test cases and constant creation of projects isn't easy. There is also too much API integration into automation tools, which is not there in ALM with UFT. Instead of setting it up as a project and using it, we set it up as a system for usability. It also lacks in the traceability aspect. For traceability, you need to use the JIRA plugin and drag traceability on JIRA, but the functionality is still quite limited. The biggest gap is mainframe testing. It would be good if I could start with mainframe testing. Manual granting of access is another issue. There is no API that I could use with another system where it is automated. There is an API for loading somebody to a project but not for adding to the application."
"It's not easy to create a custom report. It's not straightforward. A good improvement would be if there was a way to report and create a custom report without using a plugin or scripting language."
"The reports should be more user-friendly."
"Reporting could be more flexible regarding repeating reports."
 

Pricing and Cost Advice

"The option we chose was around $2,000 USD."
"Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day."
"The pricing is a little above average — it could be lower."
"It comes with a high cost."
"My advice so far, is that while it’s not quite as powerful and easy to use as UFT, its price tag more than makes up for it."
"The product is becoming more and more expensive."
"It costs a few hundred per year, but I am not sure. It is not at all expensive as compared to other tools."
"The license price for a physical machine is cheap, and for virtual machine, it is very expensive."
"I give the price a five out of ten."
"Pricing for small teams seems correct with respect to competitors."
"Use TestRail Cloud (online TR hosted server) and don't worry about maintenance or scalability. It saves a lot of cash and time."
"Its price is definitely not more. If they introduce automation, they can charge more."
"The product is not much expensive."
"Negotiate the best deal you can."
"The product has a reasonable price in terms of the features."
"My advice to others is to shop around for the best deal. Some options out there are free in cyberspace."
report
Use our free recommendation engine to learn which Test Automation Tools solutions are best for your needs.
814,649 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Computer Software Company
19%
Manufacturing Company
13%
Financial Services Firm
11%
Government
8%
Computer Software Company
22%
Manufacturing Company
10%
Financial Services Firm
10%
Healthcare Company
7%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about SmartBear TestComplete?
TestComplete has strong reporting capabilities. The reports they generate are really good.
What is your experience regarding pricing and costs for SmartBear TestComplete?
The solution's pricing is too high. On a scale from one to ten, where one is cheap and ten is expensive, I rate the solution's pricing nine and a half out of ten.
What needs improvement with SmartBear TestComplete?
The learning curve of the solution's user interface is a little high for new users.
 

Also Known As

No data available
TestRail by Gurock
 

Learn More

 

Overview

 

Sample Customers

Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.
Apple, Microsoft, Boeing, Intel, NASA, Amazon, HP, Samsung
Find out what your peers are saying about SmartBear TestComplete vs. TestRail and other solutions. Updated: April 2023.
814,649 professionals have used our research since 2012.