The automation has improved over time, allowing it to integrate with other external tools for defect tracking.
Systems Engineer at a tech services company with 51-200 employees
The automation module helped us save time in regression tests, but we needed to perform adjustments on our systems to get it to fit in.
What is most valuable?
How has it helped my organization?
The automation module helped us save time in regression tests.
What needs improvement?
It works by recording and playing a script for some part of development. However, there's no way of knowing where it failed. The developers needed help to correct these errors to script them in order to work. I noticed many changes made from version to version.
This tool is also very expensive compared to the similar tools in the market. Price needs to be affordable.
For how long have I used the solution?
I've used it since 2012.
Buyer's Guide
SmartBear TestComplete
October 2024
Learn what your peers think about SmartBear TestComplete. Get advice and tips from experienced pros sharing their opinions. Updated: October 2024.
814,763 professionals have used our research since 2012.
What was my experience with deployment of the solution?
We faced lots of code breaking, and needed to perform adjustments on our systems to get Test Complete to fit in.
What do I think about the stability of the solution?
Sometimes it runs fast, and sometimes it runs slow.
What do I think about the scalability of the solution?
No issues encountered.
How are customer service and support?
Customer Service:
They used to respond in less than 24 hours, and compared to SpiraTest, it's not that great.
Technical Support:SpiraTest took a minute to respond to any technical support issues, but Test Complete takes longer.
How was the initial setup?
It's not complex.
What about the implementation team?
It was already in place when I joined the company.
What was our ROI?
6/10.
What's my experience with pricing, setup cost, and licensing?
It's priced high, and they should look into it to make it more competitive.
Which other solutions did I evaluate?
We also looked at Selenium IDE, and chose it because it's cheaper.
What other advice do I have?
Check the price and compare to other available tools in the market and decide select the one best fits the needs.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
QA Head at a computer software company with 1,001-5,000 employees
Offers valuable regression testing and scalable with a straightforward setup
Pros and Cons
- "The solution is great as a record and playback tool. It also has valuable regression testing."
- "The solution needs more training manuals or some form of online forum for learning. It needs more documentation."
What is our primary use case?
We primarily use the solution in our data center as an application.
What is most valuable?
The solution is great as a record and playback tool. It also has valuable regression testing.
What needs improvement?
The solution needs more training manuals or some form of online forum for learning. It needs more documentation.
Some sort of troubleshooting portal should be available for users.
The solution could be faster. It would be good to increase the reusability of the code so that rerun time can be saved. It would be ideal to be able to run multiple scenarios at the same time.
Right now, in terms of recording and scripting the solution is not user-friendly. They need to improve on this.
For how long have I used the solution?
I've been using the solution for one and a half years.
What do I think about the stability of the solution?
The stability needs a bit of improvement. On the open-source version, you're not really able to check stability like you can on the paid version.
What do I think about the scalability of the solution?
The solution is definitely scalable. We have about ten licenses. We may increase usage in the future.
How are customer service and technical support?
Technical support could be better. The exact solutions we need should be provided in a more timely manner.
Which solution did I use previously and why did I switch?
We didn't previously use a different solution.
How was the initial setup?
The initial setup was straightforward. It's a straightforward installation. Deployment took about half a day.
What about the implementation team?
We handled the implementation ourselves.
What other advice do I have?
The solution is pretty good. Other options to use are TestComplete, and, for negative scenarios, Katalon Studios.
I'd rate the solution eight out of ten.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
SmartBear TestComplete
October 2024
Learn what your peers think about SmartBear TestComplete. Get advice and tips from experienced pros sharing their opinions. Updated: October 2024.
814,763 professionals have used our research since 2012.
Project Lead at Aspire Systems
A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. It is tedious to monitor the progress since the logs are available only in the tool.
Authored by Sabhari Murugan and Ujwal Unni from Aspire Systems Testing Community.
Overview
Software Quality is dependent on Test Automation for its ability to provide vast coverage with least possible resource. Having said that, automation has its own overhead and maintenance which requires additional effort. To reduce such additional effort, besides ensuring stability, robustness and increase the ROI - there are certain considerations to be focussed at different stages of the project. Here is an insight to such areas of considerations and how it can be dealt with ‘TestComplete’ – a potential test automation tool.
What is TestComplete?
Started as a tool that is preferred for Test Automation of Desktop applications, today TestComplete is one of the top competitors and a noticeable performer in the commercial market. The recent versions of the tool support Desktop, Web and mobile applications to a great extent. The tool gained its popularity due to its
- Robust and stable execution engine
- Rich in-built libraries
- Support for multiple Scripting Language
- Minimal knowledge required for beginners
- Access to System level operations like system restart, Process management and monitoring
- Cost effective, when compared with similar tools in the market
- Events (Decide what to do at - Beginning of Test, End of Test, Error encountered, etc...)
Doing it right:
Test Automation is not an area with constant or limited needs and expectations. New technologies, frequent changes, emerging software systems and new feature additions create constant challenges for tools and testers. There are certain considerations in each phase of Test Automation solution to build a suite that could provide high ROI, with least effort and maintenance. Here is a brief about some of these challenges / requirements that are more likely in Test Automation solutions that use TestComplete.
1. Frequent Changes
“Change is good” – in many ways, for instance it makes way for new features and enhances user experience. Besides, who likes to use software that has no updates? For any product to be successful, it should have the ability to change and adapt to the cutting edge technology.
Implementation stage: Framework design.
Solution:
1. Name Mapping – an object repository used to store the instance of controls. By doing so any change in the Properties of controls such as size, colour or caption can be isolated leaving the automation scripts unaffected.
2. Modularity Framework- a Test Automation framework in which all the common functions such as entering text in text box, selecting items in a combo box, clicking buttons are developed as reusable functions and used in Test Case Script
Doing so, the Test Case Script will not have any code that directly interacts with the product, the controls are referred through Name Mapping and actions are performed through the re-usable functions. Hence Test Case Script is isolated and no change will be required unless there is a change in the workflow of the product.
2. Testing Multi-Language Products
When testing applications that are released in different language versions it is crucial to test all the language versions to ensure quality.
Implementation stage: Framework design.
Solution:
1. Using an XML or ini file. The file contains a common ID for each caption and their equivalent value in the respective language. For example, below is a representation of the xml file content for the caption of Ok button in three languages.
<caption>
<id=’YesButtonCaption’>
<English=’Yes’>
<German=’Ja’>
<Finnish=’Kylla’>
</caption>
In the Test Case Script or in the reusable functions instead of directly using the caption, use a re-usable function that fetches the appropriate language text from the XML when the id and the language are passed as parameters. For example: getCaption(“YesButtonCaption”, “German”)
3. Different Levels of Execution
Test Suites may vary from less than hundred cases to more than 20,000 cases. It is not always the case that the entire test suite has to be executed; hence cases should be grouped based on features or workflows. For example automation suite of a banking application can be grouped based on features like money transfer, account creation, payments and so on to enable execution at three levels – Case level, Selected Group level and suite level.
Implementation stage: Framework design.
Solution:
1. Driver Script – Instead of matching each test Item to a test case, group them based on features and create a driver script for each feature. A driver script is just a function that calls all the test cases under the respective feature one after the other.
2. Further enhancement can be done by using XLS file to have list of Cases and a column with Yes or No values that determine if the particular case has to be executed or not.
4. Windows event log monitoring
There are scenarios where the application may seem to work fine, even when there are some issues while updating data in the database or storing data to a file. Monitoring the Windows Event log gives a better insight into such situations and helps to identify them earlier.
Implementation stage: At the time of requirement.
Solution:
1. TestComplete provides full read access to Windows event log, leveraging this feature at suite level can capture all the events logged related to the application under test during execution.
2. TestComplete also provides access to real-time resource usage of any process such as CPU usage and Memory usage, logging these details will help identifying memory leakages.
5. Memory issues due to large suites
Scalability is one essential aspect to be considered in Test Automation. TestComplete is designed to handle any number of Test Cases which means almost any Lines of Code (LOC). But having too many script units will have a huge memory foot print which accumulates over execution, which eventually leads to tool crash or other performance issues.
Implementation stage: At the time of requirement.
Solution:
1. If there are way too many Test Scripts in an Automation suite, the suite can be split into multiple smaller suites. All the smaller suites are independent of each other. In this case, when a whole suite execution is required Batch Files can be used to continue the execution from one suite to another.
6. Continuous Long-time Execution
Having around a few thousand Test Cases in a Test Suite means executions can go on continuously for days. This makes the system vulnerable for performance and memory issues.
Implementation stage: At the time of requirement.
Solution:
1. The application under test can be closed and restarted in certain intervals to free memory and have better performance
2. It is not just the application but also the tool that is in running state for too long, hence it makes a difference if the tool is restarted in some required intervals approximately once a day. This can be made possible by two ways- TestComplete’s in-built function “RebootandContinue”,which restarts the system and continues further execution or using batch file to close TestComplete after sometime into execution, let the system be idle for a while, then start TestComplete and continue the execution.
7. Automatic Report generation and mail notifications
When execution continues for days together it is tedious to monitor the progress since the logs are available only in the tool, this means either the execution has to be paused or we will have to wait till it completes to view the report or log
Implementation stage: At the time of requirement.
Solution:
1. The in-built function to Save Logs can be used at regular intervals to automatically export logs to the specified location as and when required.
2. TestComplete also supports automatic mail notifications with the logs as attachments. Once configured results will be mailed automatically as per the configuration
8. Scheduling Tests in Test Execute
SmartBear has an execution tool that is relatively less expensive than TestComplete that can be used only for placing executions. Unfortunately, TestExecute does not enable us to select the Tests that has to be executed. So one way or another TestComplete is required to modify the Tests to be executed.
Implementation stage: At the time of requirement.
Solution:
1. The .PJS and .MDS files in a Test Suite are the two XML files that stores the data about the project and Test items to be executed. Developing a simple tool to read and edit these XML files will completely eradicate the need of TestComplete in placing Executions.
Other Best Practices:
- Always ensure each test script works independently.
- Avoid test data dependencies between test scripts.
- Test data is always externalized.
- Write the test scripts from scratch instead of using record and playback.
- Always ensured test automation scripts sequence are configurable for execution.
- Frequent catch-up with Dev & Design team to understand the impact on automation scripts due to upcoming releases / changes.
- Tool & Environment upgrades should be taken up as early as possible to avoid surprises at neck of the moment.
A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. Automation Framework is the basic building block of a robust automation suite and framework design is the most important phase of an automation project. This is the phase where all the above considerations should be predicted in order to design a framework that meets all the requirements of the project and accommodate further improvements and new requirements that may arise over the course of the project. Such Anticipation, preparation and worked arounds are the way to ’Doing it right, with TestComplete’.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Test Automation Specialist at a tech services company with 501-1,000 employees
It gives us the option to work in the same suite without changing from one project to another.
What is most valuable?
- Object spy and object recognition are working very well, giving us a lot of possibilities.
- Option to work in the same suite without changing from one project to another, which is better than HP QTP/UTF.
- USEINIT feature is really nice (+/- like importing in Java)
How has it helped my organization?
It didn’t really change the way my organization works.
What needs improvement?
A test case design (schema) module would be nice.
For how long have I used the solution?
I've been using it for nine months, and my customer has been using it for three years.
What was my experience with deployment of the solution?
No issues encountered.
What do I think about the stability of the solution?
No issues encountered.
What do I think about the scalability of the solution?
No issues encountered.
How are customer service and technical support?
I just contacted them once. They responded very quickly, which is good, but the problem was on our side, so I cannot really evaluate.
Which solution did I use previously and why did I switch?
TestComplete was already in place when I arrived, but I know that they chose it because of the debugger and the fact that TestComplete recognizes more objects than QTP/UFT.
What about the implementation team?
It was done in-house.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
QA Team Lead at a tech vendor with 10,001+ employees
It supports .Net, but it needs to have templates of tests from scratch.
What is most valuable?
- Good IDE
- Support of third party controls
- Flexible in coding
- Supports .Net assemblies
- Good community
How has it helped my organization?
It's decreased the time needed for the regression testing cycle.
What needs improvement?
- Memory consumption
- Speed of interaction with controls
- Sets of prepared libraries like asserts
- Templates of tests from scratch
For how long have I used the solution?
I've used TestComplete for three years.
What do I think about the stability of the solution?
It's stable.
How are customer service and technical support?
It's good, I like the support.
Which solution did I use previously and why did I switch?
Previously was QTP. TestComplete is much more flexible light,and easy to use.
How was the initial setup?
It's quite easy to start using it. There were no problems from the beginning.
What about the implementation team?
Use JScript for quick wins. Start creating tests as small as possible.
What was our ROI?
I would say that we have now automated about 100 tests, so it saves us three days every month..
What's my experience with pricing, setup cost, and licensing?
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Easy set up and test creation but the test object repository needs improvement
Pros and Cons
- "The reporting is ready to use and doesn't require any setup."
- "The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools."
What is our primary use case?
We have what we call a UK tax related application. We are using the solution to automate some functional cases of the application. Use case wise, it's usually used more on the accounting side, which is the main purpose of the application. With the available functional testing of workflows, we are able to have accountability there.
What is most valuable?
The solution's most valuable aspect is how easy it is to create tests. For example, recording options.
The reporting is ready to use and doesn't require any setup.
We can easily create test suites without much trouble. For example, by using the UFP test, we can directly create a test suite.
What needs improvement?
The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools.
For how long have I used the solution?
I have three or four months of experience on the solution. I am an architect on technical optics. Different teams work on the project as well. I mentor them sometimes as part of my job. As far as hands-on experience goes, I can say I've only actually been using it for a few months. It hasn't been more than that.
Our company as a whole, on the other hand, has been using the solution for the better part of four or five years now.
What do I think about the stability of the solution?
We haven't had any problems with stability. It's reliable. We haven't witnessed any bugs or glitches. It doesn't seem to crash on us or to freeze.
What do I think about the scalability of the solution?
The solution is scalable. There are no issues in that sense. If a company needs to expand out the solution they can do so easily. We can add as many artifacts as we want.
I'm not sure if we have plans to increase usage in the future.
How are customer service and technical support?
I've never used technical support, so I can't speak to how good technical support is.
How was the initial setup?
The initial set up was not complex. I can say it's straightforward due to the fact that we have started out relatively small. We have gradually developed it. We continue to build it out all the time. That made it simple. This was important for us, as our team didn't have too much experience with the solution.
The deployment was handled by my IT team. I did not handle the deployment personally.
What about the implementation team?
The implementation was handled in house by our own teams.
What's my experience with pricing, setup cost, and licensing?
The pricing is reasonable. It's not too expensive as a solution. We're satisfied with the billing so far.
What other advice do I have?
I'd rate the solution seven out of ten.
If you compare it with competitors, the object identification needs to be improved. It also needs to ensure that going forward it supports new technologies. We are seeing a lot of new technologies coming up. It's important that test companies support these new advancements. If they do, then it will give testers a reason to try them out and hopefully adopt the solution. It's a win-win.
The solution is also great for language flexibility. It's important to support a number of languages to help increase user adoption.
I'd recommend this solution to other organizations. It's one of three I'd recommend for sure based on the pricing and feature offering it has. Of course, this is just a recommendation n terms of a commercial tool. Open-source tool recommendations are a whole other question.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Systems Engineer at a tech services company with 51-200 employees
It is similar to other tools like Selenium IDE where you can record and play test scripts. There is no support for mobile testing automation.
Valuable Features:
For more than a year, I've been working on automation of functional testing, regression testing using TestComplete. It is a great tool and hard to learn and use.
Pros:
- It is similar to other tools like Selenium IDE you can record and play test scripts
- Developer needed to create test scripts using VB Script, Jscript, Delphiscript etc -
- QA perspective, helps test results logs helps in finding precisely the issue where test failed
- Tests can be recorded in keyword driven or screen capture modes on some screens.
- Tests can be created for regression, functional, DDT (data driven testing), ODT(object driven testing), and so on and it is good to use it for Cross browser testing.
Cons:
- There is no support for mobile testing automation.
- QA need to depend upon developer to correct the scripts any change happens. Automated
Improvements to My Organization:
Recording the scripts and running the scripts should be available to all users.
As a QA, I would like to use it by myself using the application recording and replaying the tests.
Room for Improvement:
On some of the UI application pop up screens, Closing and opening screens etc.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Systems Test Manager at a transportation company with 1,001-5,000 employees
The latest version can be used for mobile testing, but more effective object mapping for Citrix systems is needed.
What is most valuable?
Keywords-Driven frameworks and Data-Driven Testing (DDT) methodologies used for building effective frameworks.
How has it helped my organization?
The fact that the latest version can be used for mobile testing.
What needs improvement?
A more effective way in Objects Mapping for applications built on domains/platforms such as Sliverlight, and Citrix is needed.
For how long have I used the solution?
I've used it for three years.
How are customer service and technical support?
Technical support can be improved by being more efficient in responding to questions.
Which solution did I use previously and why did I switch?
Used some other products, but this was found to be the most suited for business requirements.
How was the initial setup?
The initial step was straightforward which was made by carrying out a POC and the decision was to go for this solution.
What about the implementation team?
Following initial training by the vendor, the implementations are done in-house.
What's my experience with pricing, setup cost, and licensing?
It is value for money, and different licensing options are available.
What other advice do I have?
Fig 1: Building of Automation Testing framework.
Fig 2: Automation Testing Results
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Hello, Thank you very much for your feedback! We will use it to improve TestComplete.
Could you please elaborate on what specific aspects of the application you would like us to improve as demonstrated in figure Figure 1 and Figure 2?
Regards,
Masha
Buyer's Guide
Download our free SmartBear TestComplete Report and get advice and tips from experienced pros
sharing their opinions.
Updated: October 2024
Popular Comparisons
Tricentis Tosca
Automation Anywhere
Apache JMeter
Katalon Studio
OpenText UFT One
BrowserStack
OpenText ALM / Quality Center
Selenium HQ
Tricentis NeoLoad
Sauce Labs
BlazeMeter
Eggplant Test
Buyer's Guide
Download our free SmartBear TestComplete Report and get advice and tips from experienced pros
sharing their opinions.
Quick Links
Learn More: Questions:
- SAP GUI Testing Tool
- Which product do you prefer: Micro Focus UFT One or SmartBear TestComplete?
- Which is the best RPA solution for performance testing automation?
- What are your recommended Accessibility Testing tools (both open-source and licensed ones)?
- What is the best Automated Functional Testing Tool for Testing Pega Applications?
- What is End-to-End (E2E) test automation?
- What is the Robotic Process Automation (RPA) life cycle?
- Why is Test Automation Tools important for companies?
- What performance testing tool do you recommend for web applications?
- What is the best test automation tool for SAP?
what is the best way to convert UFT scripts to TC compatible scripts.