SmartBear TestComplete Reviews

The two most valuable features are the manual script library (projects) and Test Execute.

I set up a manual test for new functions and hand it off to a tester (who uses Test Execute). Once the new function test is completed, it is added to regression for all future versions. This has allowed us to make significant product changes, new server deployments, and hosting changes with high confidence.

From my understanding, the product has changed significantly since v5 was released. I don’t know that it has necessarily improved, for my needs. If I worked in a different shop, then I might need an entirely different feature set compared to what I use now.

We've used it since May 2007.

No issues encountered.

No issues encountered.

No issues encountered.

I can't recall the last time I needed support.

I can't recall the last time I needed support.

Typically, I used my own method for automation (scripting tool, test data stored in SQL, results stored in SQL, result comparison to baseline with custom script). At the time I purchased TestComplete, the script recording tool wasn’t that great and I didn’t have the skills to write my own scripts. Over time, I used a different automation program and picked up the syntax for its scripting language. The product is currently called WinAutomation, though it had a different name at the time, that I can’t recall.

The initial setup was very straightforward. I installed the main app on my local workstation and test execute on five workstations available for testers. We use a shared network drive to store tests.

I installed everything myself. It was very straightforward. The real key is knowing what you need for your environment and to adapt the available tools.

It’s hard to sell testing tools, especially in a company that is owned by someone who thinks of software as a mule to pull his cart. It was a very tough sell and it is hard to quantify in dollars the money saved. However, I can say, with confidence, that there have been over one hundred bugs caught through regression since the software was purchased, which is a great track record for a team this size.

To me, the Object Browser has proven to be the most valuable feature of this product. This tab (and its associated tools) allows the test developer to locate objects within the application to be tested, and retrieve properties and methods that can be used in scripting. This saves a significant amount of time over referring to the application source code for such information. It also makes use of the recording feature periodically to rough out scripts for new tests. The recorded scripts are very fragile, but they can provide a good base for writing stable scripts.

While our test automation initiative is still in its early stages, TestComplete is allowing us to script and run complex business process tests against our desktop applications. Being able to create smaller modules that can be referenced by other scripts or keyword tests allows for a flexible and easy to maintain test suite.

A major part of the product that needs improvement is the licenser. This piece of the application is known to have issues installing that requires manual, command-line modifications to correct. While this doesn’t occur with all installations, it has been a frequent problem for me. The larger issue with the licenser is that it is incompatible with virtual machines. The applications themselves will run on physical or virtual machines, but they must call out to a physical server which controls the licenses for them.

The latest version of TestComplete has introduced support for Git, but my initial impression is that it still needs some work before it can replace my VCS tool.

The product has been in use at the company for about two years and I have been actively using it for about 8 months.

I haven’t experienced issues that I would attribute directly to the application. There have been some issues with timing and other common problems, but many of those have been corrected through modifications to the tests and the environment.

My experience with their technical support has been less than satisfactory, but all of my issues were resolved. Of the technicians I have worked with, two of them were very knowledgeable and one of them did little more than email knowledge base articles that had been ruled out in the original issue report. The latter tech had the additional problem of being about 12 hours out of sync so a full day was lost with every email exchange. I received prompt responses and quick resolutions from both of the knowledgeable technicians I have dealt with although one of them was rather gruff and left me feeling like I was bothering her. Overall, SmartBear needs to work on their customer service and technical support.

This application did not replace a previous automation effort.

When deploying new installations of TestComplete, I ran into multiple issues with the licensing engine. The first time I experienced the problem, I contacted support and was directed to a knowledge base article (which I had been unable to find while searching for the error message). I have since bookmarked the article and reference each time I need to do a new install. Once the licenser has installed properly, the configuration and licensing goes smoothly.

The implementation was performed in-house, and was quite easy despite the issues with the licenser. The best advice I can offer would be to plan out your testing environment and requirements before you begin. If you plan to use virtual machines for your test systems, make sure you also have a physical box to run the licenser that is dedicated to the lab. Do not attempt to use someone’s PC as it may not always be available when the test machine are trying to run.

We have not performed an ROI analysis.

SmartBear’s product licenses are reasonably priced and generally in line with similar companies and products.

I reviewed several products before deciding to continue our automation efforts using TestComplete. Among the products reviewed were Telerik TestStudio, Testing Anywhere, and Ranorex.

Mainly we use the solution for when we need to install or deploying in different applications, if necessary. We use it for testing for the most part.

It's a good solution. It works in Windows Operating System.

The automation is very good.

We can test an application across many programs with ease. We can test, for example, on Internet Explorer, or Mozilla, or Google Chrome, et cetera. 

It's very helpful that, when you have a recording in one application, in one navigator, you can replay every step in different applications. You don't need to do any different testing in different programs, you only have to do one test and replicate it in different programs.

The initial setup is easy.

The solution has a very nice interface.

The one issue we have is that the tool was installed in a local machine. Now, it's more popular when you use tools that work in online environments. It would be better if it was available on the cloud. We'd like to access it on remote desktops. I don't want to have to install it on every machine.

It would be ideal if the solution could offer one subscription for time and one for life. Right now, when you buy the solution, you need to pay for one solution. You receive one set up and you install it and it's just in that one machine. It would be ideal if they could offer one subscription where you can connect to different machines with a group subscription. Or maybe they could offer different pricing. They need to be more flexible.

I've only used the solution for maybe one or two months at this point. It hasn't been too long. It's relatively new.

The solution is stable. We didn't run into any issues. It has been good so far.

The technical support is good. They are very responsive and helpful. We are quite satisfied with eh level of support they give us.

The initials setup is not complex at all. It's pretty straightforward. It's easy. A company shouldn't have any problems with deployment.

That said, it might be hard for a normal user. You need to do a few technical things. It helps to have a bit of knowledge on your side. Otherwise, you may run into issues.

You only need one person to set everything up.

We handled the deployment ourselves. We didn't need the assistance of an integrator or consultant.

The solution doesn't have a very flexible pricing model.

I didn't look at too many other options, however, this one had a special advantage in that it works on the Windows Operating System.

We're just customers and end-users.

I'd advise new users to take a look at what they need. They need to know if they need something that can test on desktop or on mobile, et cetera. This is a good solution if you need something that works with Windows, for example.

I would rate the solution eight out of ten.

It's cross platform automation capabilities specially ranging across web, UNIX (via putty), and other systems.

TestComplete replaced QTP as the preferred choice of tool for the organization. It is much faster, works better across technologies (esp. Flex based UI) and is better compatible with newer technologies directly out of the box. We could have our automation engineers create tests more easily. Also, we were able to set-up lab machines to enable distributed runs for more applications in a shared environment.

The code editor, though following eclipse-style, is still a work in progress and gives a very poorly formatted code once viewed via other editing tools like Notepad++. Performance is another aspect which can be improved.

I've used it for one and a half years.

TestComplete was fairly easy to scale once we had the licenses in place for the organization.

We used tech support for some specific third-party grids we had to automate. The support was average and we ended up creating our own automation solution for that piece of automation.

We previously used QTP, and later moved to TestComplete as it provided more out of the box support for newer technologies like Flex and AIR.

Initial setup was very straightforward. We did face some hiccups in license procurement however, once licenses were procured the process from there was quite smooth.

We had a mixed team. The implementation was smooth overall and requires a few skilled automation experts to oversee the transition/initial implementation.

We achieved ROI in eight months from the start of implementation. Get a good automation architect to implement a good ROI directed framework. It is very easy to lose direction during a mmigration.

• QTP
• Selenium

It's easy to work with and doesn’t take much to get it setup to start working with it.

In my previous three positions, there was not a test automation solution in place. I was given the opportunity to explore options. Once I chose a solution, we were able to implement TestComplete and were able to automate about 90% of the manual testing that was done prior to implementing TestComplete.

This product continually improves and in v11, they now have Python support. This was something that I wanted and they provided it the latest version.

I've used it for 11 years.

I have not run into any issues with deployment.

I have not run into any issues with stability.

I have not run into any issues with scalability.

Another one of the many reasons why I chose TestComplete. The level of customer service and technical support can’t be beat. SmartBear always answers my questions within 24-48 hours.

They have a great website, help and forums that also help in finding the answers I need in a timely manner.

It was very straightforward which was another reason why I went with TestComplete. I was able to use the demo version (when I was searching for options) to create tests easily, so when I had to demonstrate the product to my management teams, they could easily see why TestComplete was the product for us.

The implementation was done by me. The best recommendation is to read the help guide, especially if you are using the product in different ways, like floating licenses. This is where the license server is on one machine and people have to access TestComplete that are not local. However, if you run into any issues, the customer support department is there to help in any way they can.

My ROI has been the fact that it takes less time to run all of the tests that were done previously. Prior to TestComplete, it would take over three weeks to run all of the tests that needed to be run for a release. After TestComplete, we have been able to reduce that time to less than one week.

One of the main reasons I went with TestComplete, besides the information that I already provided, was the cost for TestComplete and TestExecute. They make it very easy for large to small companies to implement without large costs. The licenses are broken down the amount of users that need to use it and they also have the option of floating licenses.

I looked at various options like QuickTest Professional, Rational Functional Tester, and SilkTest.

The test tree makes for quick work of choosing which tests you want to run at any time. Whether it's all or nothing, I can re-run parts of a test suite without having to start over again.

The Object Spy has surpassed any other "spy" tools I have ever used. It allows me to input parameters without having to write a quick Visual Basic script around it to try it. Diving deep into object trees can be made a breeze with the search function.

I would like to see improved language support, with Python being my

first choice. Visual Basic is outdated, where C# has been its

replacement for years.

This solution has been in use for almost three years, since I started working at this company. It has been mostly stable with static test cases, and only minor maintenance. The updates typically are changing search criteria. We upgraded from v9 about one year ago.

Technical support has been good. There was an issue where certain installations of TestComplete would give an ambiguous error on startup. After teleconferencing with one of their support specialists, we were able to solve it. Forum support on their community site has also been good.

No previous solution I know of was implemented prior to my employment here. Attempting to implement the same solution in UFT would have been nearly impossible.

The initial setup was straightforward. Licensing was simple, and getting the initial object mapping was painless. Only a high-level base set of object mapping was done.

The implementation was done entirely in-house. The projects were made from scratch, where function libraries & test suites were made from existing manual test cases.

• High UI technologies coverage
• Direct access to object internal properties and methods

Faster feedback on any product changes and uncovered problems before full regression starts. That can save a lot of time.

Scripting languages should be extended to support more popular programming languages like Java, C#, Ruby, Python. Many other vendors have such support or moved into such a direction. e.g. SilkTest has OpenAgent since 2008 with support of Java, C#. HP recently published LeanFT which integrates with popular development environments (Eclipse, Visual Studio). IBM solutions were initially designed for Java. Squish supports multiple languages. So, TestComplete should have support of such programming languages apart from VBScript and JScript modifications.

About three years onwards.

TestComplete had problems during long hours runs. In some cases it could crash without leaving any logs. Also, if there were some unhanded exception it could drop the error message waiting for user input. This is not really acceptable for nightly runs and required human assistance. But this problem is rather language-specific

I never had to speak to them.

I never had to speak to them.

The setup is pretty easy.

We used a vendor team. My advice for implementation is only one, follow the best practices (some of them are available on the vendor site). They were formulated based on real experiences.

ROI was never calculated explicitly, but normally the ROI point was expected to be reached after three to five months of the project.

Mainly, when we did an evaluation and comparison, TestComplete had two big advantages -

• Good technology support
• Relatively low price in comparison to other vendor tools

I'm currently are trying to apply it to our software application.

We use it for testing Windows applications for CUTE, based on the CUTE framework. For now, that's all we use it for because Complete does not suggest any other options that are appropriate for us. By that, I mean it doesn't seem to work with Mac OS, Android OS, and iOS.

I only have experience with Windows, so I find all the best features are for that operating system.

Complete works perfectly with CUTE. That includes all dialogues, right-click menus, or system dialogues, etc., which are handled well.

The solution has a great feature called macro recording. With it, I can make it into a macro in a few of the languages Complete suggests. This is really useful for me, personally.

The solution needs to extend the possibilities so that we can test on other operating systems, platforms and publications for Android as well as iOS.

Right now, Complete can test only on native Android and native iOS applications.

I've been using the solution for about a month. 

This tool is quite stable. I had only one crash, and I sent a report for this crash to technical support. Everything else seems to work perfectly. Aside from the one issue, we haven't had any other problems.

I don't have any experience, unfortunately, with scalability. We use just one instance on one machine so I've never attempted to scale the solution. There's just one user on the platform right now.

I am not sure if our organization will extend its usage in the future. We have a strong need to make this work on other platforms. We may switch to a different tool.

I do have some experience with technical support. I've found a few answers to my issues on Smart BF5. I've also had assistance with the support engineers. So far, I've been satisfied with the level of support I have been able to receive. 

I have used other solutions, including ALTA. It has good UI, but I don't know if you are really able to directly compare it to this solution. 

The initial setup is straightforward. We found it simple and not too complex. Our team didn't run into any issues.

For us, deployment took two to three days.

We didn't use an integrator, reseller or consultant for the deployment. We handled the entire process ourselves.

I'm not sure if there are licensing costs involved in the solution. We simply bought the product outright and started using it.

I'm currently working with the latest version of the solution.

My advice to others is to just decide if this tool is usable for your requirements. I spent a lot of time developing some tests, but then I understood that we actually needed more platforms. That's why we will switch to another tool. That's also why it's important to check your organization's requirements. Otherwise, like us, you may need to switch.

I'd rate the solution seven out of ten.