Try our new research platform with insights from 80,000+ expert users
PeerSpot user
Test Automation Lead/Consultant at Aspire Systems
Real User
Download
It supports various and different versions of browsers in web testing, but it's slightly unstable with longer executions of automation scripts.
Pros and Cons
  • "Test items, project variables helps in managing automation suite and scheduling execution."
  • "Increased performance with less memory and CPU usage."

What is our primary use case?

Evaluated for automating regression suite for windows desktop application and found testcomplete suits well in the aspect of identifying most of the objects and in few other parameters.

How has it helped my organization?

Good test coverage through automation and provides unique solutions to the most of automation challenges (e.g. comparison of images).

What is most valuable?

- Object spy eases the object identification method.

- TextObject recognition feature of this product exposes most of the objects to perform the actions.

- Test items, project variables helps in managing automation suite and scheduling execution.

- Jenkins integration to schedule executions.

- Integrated with selenium webdriver.

- Conversion of JScript project to Javascript project

- Integrated to Environment manager

- Support to read text from PDF

What needs improvement?

- Increased performance with less memory and cpu usage.

- Supports various and different versions of browsers in web testing.

- Support to Windows mobile application. (Native iOS and Android apps are supported currently.)

Buyer's Guide
SmartBear TestComplete
November 2024
Free Report: SmartBear TestComplete Reviews and More
Learn what your peers think about SmartBear TestComplete. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
DOWNLOAD NOW
816,660 professionals have used our research since 2012.

For how long have I used the solution?

More than five years.

How are customer service and support?

Customer Service:

Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

Technical Support:

Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

What about the implementation team?

In-house team is used for implementing framework for automation through TestComplete. Tool must be explored completely and knowing its unique features is more important before implementation as it has lot of inbuilt features.

What's my experience with pricing, setup cost, and licensing?

TestComplete now have come up with three modules (Web, Desktop & Mobile), so based on the type of product for automation, it is adequate to purchase the required module. Node-locked and floating license are available which could be decided based on the team composition and work strategy. Current license pricing is available at http://smartbear.com/product/testcomplete/pricing/

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Marvin Reekie - PeerSpot reviewer
Marvin ReekieQuality Assurance Analyst at a legal firm with 1,001-5,000 employees
Real User
Report as inappropriate

In the process of evaluating tools to help improve the turn around for desktop testing. Looking at Testcomplete and Ranorex at this time. Still in the decision making process.

Like(0)Reply
See all 2 comments
PeerSpot user
Systems Engineer at a tech services company with 51-200 employees
Consultant
Download
The automation module helped us save time in regression tests, but we needed to perform adjustments on our systems to get it to fit in.

Valuable Features

The automation has improved over time, allowing it to integrate with other external tools for defect tracking.

Improvements to My Organization

The automation module helped us save time in regression tests.

Room for Improvement

It works by recording and playing a script for some part of development. However, there's no way of knowing where it failed. The developers needed help to correct these errors to script them in order to work. I noticed many changes made from version to version.

This tool is also very expensive compared to the similar tools in the market. Price needs to be affordable.

Use of Solution

I've used it since 2012.

Deployment Issues

We faced lots of code breaking, and needed to perform adjustments on our systems to get Test Complete to fit in.

Stability Issues

Sometimes it runs fast, and sometimes it runs slow.

Scalability Issues

No issues encountered.

Customer Service and Technical Support

Customer Service:

They used to respond in less than 24 hours, and compared to SpiraTest, it's not that great.

Technical Support:

SpiraTest took a minute to respond to any technical support issues, but Test Complete takes longer.

Initial Setup

It's not complex.

Implementation Team

It was already in place when I joined the company.

ROI

6/10.

Pricing, Setup Cost and Licensing

It's priced high, and they should look into it to make it more competitive.

Other Solutions Considered

We also looked at Selenium IDE, and chose it because it's cheaper.

Other Advice

Check the price and compare to other available tools in the market and decide select the one best fits the needs.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Buyer's Guide
SmartBear TestComplete
November 2024
Free Report: SmartBear TestComplete Reviews and More
Learn what your peers think about SmartBear TestComplete. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
DOWNLOAD NOW
816,660 professionals have used our research since 2012.
reviewer1233750 - PeerSpot reviewer
QA Head at a computer software company with 1,001-5,000 employees
Real User
Download
Offers valuable regression testing and scalable with a straightforward setup
Pros and Cons
  • "The solution is great as a record and playback tool. It also has valuable regression testing."
  • "The solution needs more training manuals or some form of online forum for learning. It needs more documentation."

What is our primary use case?

We primarily use the solution in our data center as an application.

What is most valuable?

The solution is great as a record and playback tool. It also has valuable regression testing.

What needs improvement?

The solution needs more training manuals or some form of online forum for learning. It needs more documentation. 

Some sort of troubleshooting portal should be available for users.

The solution could be faster. It would be good to increase the reusability of the code so that rerun time can be saved. It would be ideal to be able to run multiple scenarios at the same time.

Right now, in terms of recording and scripting the solution is not user-friendly. They need to improve on this.

For how long have I used the solution?

I've been using the solution for one and a half years.

What do I think about the stability of the solution?

The stability needs a bit of improvement. On the open-source version, you're not really able to check stability like you can on the paid version.

What do I think about the scalability of the solution?

The solution is definitely scalable. We have about ten licenses. We may increase usage in the future.

How are customer service and technical support?

Technical support could be better. The exact solutions we need should be provided in a more timely manner.

Which solution did I use previously and why did I switch?

We didn't previously use a different solution.

How was the initial setup?

The initial setup was straightforward. It's a straightforward installation. Deployment took about half a day.

What about the implementation team?

We handled the implementation ourselves.

What other advice do I have?

The solution is pretty good. Other options to use are TestComplete, and, for negative scenarios, Katalon Studios.

I'd rate the solution eight out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
PeerSpot user
Project Lead at Aspire Systems
Real User
A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. It is tedious to monitor the progress since the logs are available only in the tool.

Authored by Sabhari Murugan and Ujwal Unni from Aspire Systems Testing Community.

Overview

Software Quality is dependent on Test Automation for its ability to provide vast coverage with least possible resource. Having said that, automation has its own overhead and maintenance which requires additional effort. To reduce such additional effort, besides ensuring stability, robustness and increase the ROI - there are certain considerations to be focussed at different stages of the project. Here is an insight to such areas of considerations and how it can be dealt with ‘TestComplete’ – a potential test automation tool.

What is TestComplete?

Started as a tool that is preferred for Test Automation of Desktop applications, today TestComplete is one of the top competitors and a noticeable performer in the commercial market. The recent versions of the tool support Desktop, Web and mobile applications to a great extent. The tool gained its popularity due to its

  • Robust and stable execution engine
  • Rich in-built libraries
  • Support for multiple Scripting Language
  • Minimal knowledge required for beginners
  • Access to System level operations like system restart, Process management and monitoring
  • Cost effective, when compared with similar tools in the market
  • Events (Decide what to do at - Beginning of Test, End of Test, Error encountered, etc...)

Doing it right:

Test Automation is not an area with constant or limited needs and expectations. New technologies, frequent changes, emerging software systems and new feature additions create constant challenges for tools and testers. There are certain considerations in each phase of Test Automation solution to build a suite that could provide high ROI, with least effort and maintenance. Here is a brief about some of these challenges / requirements that are more likely in Test Automation solutions that use TestComplete.

1. Frequent Changes

“Change is good” – in many ways, for instance it makes way for new features and enhances user experience. Besides, who likes to use software that has no updates? For any product to be successful, it should have the ability to change and adapt to the cutting edge technology.

Implementation stage: Framework design.

Solution:

1. Name Mapping – an object repository used to store the instance of controls. By doing so any change in the Properties of controls such as size, colour or caption can be isolated leaving the automation scripts unaffected.

2. Modularity Framework- a Test Automation framework in which all the common functions such as entering text in text box, selecting items in a combo box, clicking buttons are developed as reusable functions and used in Test Case Script

Doing so, the Test Case Script will not have any code that directly interacts with the product, the controls are referred through Name Mapping and actions are performed through the re-usable functions. Hence Test Case Script is isolated and no change will be required unless there is a change in the workflow of the product.

2. Testing Multi-Language Products

When testing applications that are released in different language versions it is crucial to test all the language versions to ensure quality.

Implementation stage: Framework design.

Solution:

1. Using an XML or ini file. The file contains a common ID for each caption and their equivalent value in the respective language. For example, below is a representation of the xml file content for the caption of Ok button in three languages.

<caption>
                <id=’YesButtonCaption’>                
                <English=’Yes’>
                <German=’Ja’>
                <Finnish=’Kylla’>
</caption>

In the Test Case Script or in the reusable functions instead of directly using the caption, use a re-usable function that fetches the appropriate language text from the XML when the id and the language are passed as parameters. For example: getCaption(“YesButtonCaption”, “German”)

3. Different Levels of Execution

Test Suites may vary from less than hundred cases to more than 20,000 cases. It is not always the case that the entire test suite has to be executed; hence cases should be grouped based on features or workflows. For example automation suite of a banking application can be grouped based on features like money transfer, account creation, payments and so on to enable execution at three levels – Case level, Selected Group level and suite level.

Implementation stage: Framework design.

Solution:

1. Driver Script – Instead of matching each test Item to a test case, group them based on features and create a driver script for each feature. A driver script is just a function that calls all the test cases under the respective feature one after the other.

2. Further enhancement can be done by using XLS file to have list of Cases and a column with Yes or No values that determine if the particular case has to be executed or not.

4. Windows event log monitoring

There are scenarios where the application may seem to work fine, even when there are some issues while updating data in the database or storing data to a file. Monitoring the Windows Event log gives a better insight into such situations and helps to identify them earlier.

Implementation stage: At the time of requirement.

Solution:

1. TestComplete provides full read access to Windows event log, leveraging this feature at suite level can capture all the events logged related to the application under test during execution.

2. TestComplete also provides access to real-time resource usage of any process such as CPU usage and Memory usage, logging these details will help identifying memory leakages.

5. Memory issues due to large suites

Scalability is one essential aspect to be considered in Test Automation. TestComplete is designed to handle any number of Test Cases which means almost any Lines of Code (LOC). But having too many script units will have a huge memory foot print which accumulates over execution, which eventually leads to tool crash or other performance issues.

Implementation stage: At the time of requirement.

Solution:

1. If there are way too many Test Scripts in an Automation suite, the suite can be split into multiple smaller suites. All the smaller suites are independent of each other. In this case, when a whole suite execution is required Batch Files can be used to continue the execution from one suite to another.

6. Continuous Long-time Execution

Having around a few thousand Test Cases in a Test Suite means executions can go on continuously for days. This makes the system vulnerable for performance and memory issues.

Implementation stage: At the time of requirement.

Solution:

1. The application under test can be closed and restarted in certain intervals to free memory and have better performance

2. It is not just the application but also the tool that is in running state for too long, hence it makes a difference if the tool is restarted in some required intervals approximately once a day. This can be made possible by two ways- TestComplete’s in-built function “RebootandContinue”,which restarts the system and continues further execution or using batch file to close TestComplete after sometime into execution, let the system be idle for a while, then start TestComplete and continue the execution.

7. Automatic Report generation and mail notifications

When execution continues for days together it is tedious to monitor the progress since the logs are available only in the tool, this means either the execution has to be paused or we will have to wait till it completes to view the report or log

Implementation stage: At the time of requirement.

Solution:

1. The in-built function to Save Logs can be used at regular intervals to automatically export logs to the specified location as and when required.

2. TestComplete also supports automatic mail notifications with the logs as attachments. Once configured results will be mailed automatically as per the configuration

8. Scheduling Tests in Test Execute

SmartBear has an execution tool that is relatively less expensive than TestComplete that can be used only for placing executions. Unfortunately, TestExecute does not enable us to select the Tests that has to be executed. So one way or another TestComplete is required to modify the Tests to be executed.

Implementation stage: At the time of requirement.

Solution:

1. The .PJS and .MDS files in a Test Suite are the two XML files that stores the data about the project and Test items to be executed. Developing a simple tool to read and edit these XML files will completely eradicate the need of TestComplete in placing Executions.

Other Best Practices:

  • Always ensure each test script works independently.
  • Avoid test data dependencies between test scripts.
  • Test data is always externalized.
  • Write the test scripts from scratch instead of using record and playback.
  • Always ensured test automation scripts sequence are configurable for execution.
  • Frequent catch-up with Dev & Design team to understand the impact on automation scripts due to upcoming releases / changes.
  • Tool & Environment upgrades should be taken up as early as possible to avoid surprises at neck of the moment.

A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. Automation Framework is the basic building block of a robust automation suite and framework design is the most important phase of an automation project. This is the phase where all the above considerations should be predicted in order to design a framework that meets all the requirements of the project and accommodate further improvements and new requirements that may arise over the course of the project. Such Anticipation, preparation and worked arounds are the way to ’Doing it right, with TestComplete’.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
it_user340992 - PeerSpot reviewer
Test Automation Specialist at a tech services company with 501-1,000 employees
Consultant
Download
It gives us the option to work in the same suite without changing from one project to another.

What is most valuable?

  • Object spy and object recognition are working very well, giving us a lot of possibilities.
  • Option to work in the same suite without changing from one project to another, which is better than HP QTP/UTF.
  • USEINIT feature is really nice (+/- like importing in Java)

How has it helped my organization?

It didn’t really change the way my organization works.

What needs improvement?

A test case design (schema) module would be nice.

For how long have I used the solution?

I've been using it for nine months, and my customer has been using it for three years.

What was my experience with deployment of the solution?

No issues encountered.

What do I think about the stability of the solution?

No issues encountered.

What do I think about the scalability of the solution?

No issues encountered.

How are customer service and technical support?

I just contacted them once. They responded very quickly, which is good, but the problem was on our side, so I cannot really evaluate.

Which solution did I use previously and why did I switch?

TestComplete was already in place when I arrived, but I know that they chose it because of the debugger and the fact that TestComplete recognizes more objects than QTP/UFT.

What about the implementation team?

It was done in-house.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
it_user323649 - PeerSpot reviewer
QA Team Lead at a tech vendor with 10,001+ employees
Vendor
Download
It supports .Net, but it needs to have templates of tests from scratch.

What is most valuable?

  • Good IDE
  • Support of third party controls
  • Flexible in coding
  • Supports .Net assemblies
  • Good community

How has it helped my organization?

It's decreased the time needed for the regression testing cycle.

What needs improvement?

  • Memory consumption
  • Speed of interaction with controls
  • Sets of prepared libraries like asserts
  • Templates of tests from scratch

For how long have I used the solution?

I've used TestComplete for three years.

What do I think about the stability of the solution?

It's stable.

How are customer service and technical support?

It's good, I like the support.

Which solution did I use previously and why did I switch?

Previously was QTP. TestComplete is much more flexible light,and easy to use.

How was the initial setup?

It's quite easy to start using it. There were no problems from the beginning.

What about the implementation team?

Use JScript for quick wins. Start creating tests as small as possible.

What was our ROI?

I would say that we have now automated about 100 tests, so it saves us three days every month..

What's my experience with pricing, setup cost, and licensing?

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
RY
QA Architect at a computer software company with 5,001-10,000 employees
Real User
Download
Easy set up and test creation but the test object repository needs improvement
Pros and Cons
  • "The reporting is ready to use and doesn't require any setup."
  • "The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools."

What is our primary use case?

We have what we call a UK tax related application. We are using the solution to automate some functional cases of the application. Use case wise, it's usually used more on the accounting side, which is the main purpose of the application. With the available functional testing of workflows, we are able to have accountability there.

What is most valuable?

The solution's most valuable aspect is how easy it is to create tests. For example, recording options.

The reporting is ready to use and doesn't require any setup.

We can easily create test suites without much trouble. For example, by using the UFP test, we can directly create a test suite. 

What needs improvement?

The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools.

For how long have I used the solution?

I have three or four months of experience on the solution. I am an architect on technical optics. Different teams work on the project as well. I mentor them sometimes as part of my job. As far as hands-on experience goes, I can say I've only actually been using it for a few months. It hasn't been more than that.

Our company as a whole, on the other hand, has been using the solution for the better part of four or five years now.

What do I think about the stability of the solution?

We haven't had any problems with stability. It's reliable. We haven't witnessed any bugs or glitches. It doesn't seem to crash on us or to freeze.

What do I think about the scalability of the solution?

The solution is scalable. There are no issues in that sense. If a company needs to expand out the solution they can do so easily. We can add as many artifacts as we want.

I'm not sure if we have plans to increase usage in the future.

How are customer service and technical support?

I've never used technical support, so I can't speak to how good technical support is.

How was the initial setup?

The initial set up was not complex. I can say it's straightforward due to the fact that we have started out relatively small. We have gradually developed it. We continue to build it out all the time. That made it simple. This was important for us, as our team didn't have too much experience with the solution. 

The deployment was handled by my IT team. I did not handle the deployment personally.

What about the implementation team?

The implementation was handled in house by our own teams.

What's my experience with pricing, setup cost, and licensing?

The pricing is reasonable. It's not too expensive as a solution. We're satisfied with the billing so far.

What other advice do I have?

I'd rate the solution seven out of ten.

If you compare it with competitors, the object identification needs to be improved. It also needs to ensure that going forward it supports new technologies. We are seeing a lot of new technologies coming up. It's important that test companies support these new advancements. If they do, then it will give testers a reason to try them out and hopefully adopt the solution. It's a win-win.

The solution is also great for language flexibility. It's important to support a number of languages to help increase user adoption.

I'd recommend this solution to other organizations. It's one of three I'd recommend for sure based on the pricing and feature offering it has. Of course, this is just a recommendation n terms of a commercial tool. Open-source tool recommendations are a whole other question.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
PeerSpot user
Systems Engineer at a tech services company with 51-200 employees
Consultant
Download
It is similar to other tools like Selenium IDE where you can record and play test scripts. There is no support for mobile testing automation.

Valuable Features:

For more than a year, I've been working on automation of functional testing, regression testing using TestComplete. It is a great tool and hard to learn and use. 
 
Pros:
- It is similar to other tools like Selenium IDE you can record and play test scripts 

- Developer needed to create test scripts using VB Script, Jscript, Delphiscript etc - 
- QA perspective, helps test results logs helps in finding precisely the issue where test failed
- Tests can be recorded in keyword driven or screen capture modes on some screens.
- Tests can be created for regression, functional, DDT (data driven testing), ODT(object driven testing), and so on and it is good to use it for Cross browser testing.
 
Cons:
- There is no support for mobile testing automation.

- QA need to depend upon developer to correct the scripts any change happens. Automated

Improvements to My Organization:

Recording the scripts and running the scripts should be available to all users.

As a QA, I would like to use it by myself using the application recording and replaying the tests.

Room for Improvement:

On some of the UI application pop up screens, Closing and opening screens etc.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
it_user242190 - PeerSpot reviewer
it_user242190Test Manager with 501-1,000 employees
Vendor
Report as inappropriate

what is the best way to convert UFT scripts to TC compatible scripts.

Like(0)Reply
Buyer's Guide
Download our free SmartBear TestComplete Report and get advice and tips from experienced pros sharing their opinions.
Updated: November 2024
DOWNLOAD NOW
Buyer's Guide
Download our free SmartBear TestComplete Report and get advice and tips from experienced pros sharing their opinions.
Buyer's Guide
SmartBear TestComplete
November 2024
DOWNLOAD NOW
Quick Links
Learn More: Questions: