SmartBear TestComplete Reviews

The opportunity to work with DevExpress and WPF objects.

• Approximately, we get new build once per week. So I need to provide smoke testing for it. In case of manual tests execution, it is needed 48 hours (six working days) to smoke it. With TestComplete, we need just eight hours (or even less - if we're using multiple machines)
• Stability and Stress testing. It is impossible to test our applications manually for 48 hours (two full days) continuously. With TestComplete, it is possible

• Better stability, as sometimes, TestComplete crashes when attempting to delete over 10 logs
• Support for the latest versions of DevExpress
• More comfortable XML editor (like in Notepad++)
• A better script editor. I will be happy, if the TestComplete editor would contain a design like the one in Visual Studio or Eclipse - errors and unusable variables being highlighte, and refactoring opporunities
• Own log viewer in TestExecute. For now, it is possible to open it only within a browser, which is not very good, because if the log is 2GB or more, it will take 10-20 minuts to open with Internet Explorer

I've used it for three years, including, v7, & v9 with TestExecute v9, and it's been in use on the project since 2010.

When we migrated from v7 to v9, we encountered a problem with using recursive code; in DelphiScript recursion was completely broken.

Sometimes, TestComplete crashes when attempting to delete over 10 logs.

It's acceptable.

It's acceptable.

I tried Visual Studio 2010 (Coded UI Tests feature). The tool was changed, because TC is cheaper and more acceptable for those apps testing. But for now, this project doesn't use QA automation and I am working in another project

The most valuable features of the SmartBear TestComplete are self-healing, they reduce the maintenance required. The different languages SmartBear TestComplete supports are good because some of our libraries are written in Python, JavaScript, and C#. It's very easy to put them all under one project and use them. The are other features that SmartBear TestComplete has but the competition widely has them as well.

In SmartBear TestComplete the integration with Jenkins could be easier. Additionally, some of the controls could have better customization options. For example, if a grid is used and it contains multiple controls within it, it can be a checkbox, radio button, or any kind of control, the way the Object Spy is operating currently there is a lot of room for improvement.

There are a lot of improvements that are required on the Object Spy, such as how we find the locators. That is a feature which I would love to see. 

It is currently very tightly coupled to the SmartBear ID. There should be a way that they could support the integration with external libraries and make it flexible to integrate with open-source frameworks. This would be a good option to add.

I have been using SmartBear TestComplete for approximately eight years.

SmartBear TestComplete is a stable solution.

When it comes to customization controls the scalability of SmartBear TestComplete is limited. However, the assets are scalable. It's easy to scale across the solutions within the same limited functionality.

The technical support from SmartBear TestComplete is good.

The initial setup of SmartBear TestComplete is easy. The process is almost instant, it is very easy.

The price of SmartBear TestComplete could be less. The main challenge is when it comes to node-locked. They should use a subscription model, such as a monthly-based subscription or, a quarterly-based subscription. Their floating license is very expensive, and this high price should be reduced or provide, at a minimum, a subscription model.

When it comes to automation, the fundamental aspects are the stabilization factor, how easy it is to automate, how much is the coverage, and how much are you able to cover with respect to an application. There will be always areas that cannot be automated. The coverage, stabilization, and ease to automate are the areas that have to be improved. The better the pricing and improved pricing models, the more it will attract more customers. 

I rate SmartBear TestComplete a seven out of ten.

TestComplete is simple, it's a very easy-to-use tool. It offers credibility and value for the money. A basic license is roughly $4,000 and you can add flexible components on top of it. 

If you want desktop testing, you can add-on a desktop license. If you want web testing, you're going to want to add-on a web license — there's a lot of flexibility, you just need to pay for what you use. There are no irritating subscription models.

People are able to quickly use the platform and with a variety of scripting languages, including Python and JavaScript — all the modern scripting languages are supported. It doesn't just rely on VB script like UFT.

It's very flexible and robust in that way. I have seen many of our clients quickly adopting the tool with all the scripting languages. 

Recently, they've been building a lot of futuristic features, for example, AI Self-healing is one of the interesting features where they try to improve and cut down on maintenance by automatically correcting the arbiter. That's a really cool feature for keeping your object repositories up to date, and it can considerably bring down or control your maintenance costs to some extent, at least as far as the object repository goes.

They also have some intelligent OCR features. They have a mini device cloud, for example, which allows you to run a testing tool and recently started supporting X spot. That actually goes well with selenium. You can reuse some of the scripts with other frameworks. They also acquired CucumberStudio a few years back. They have combined HipTest and Cucumber into one capability — CucumberStudio —, which is a great integration to TestComplete, that really becomes very seamless.

They're working on many features. Of course, the roadmap is not news to me, but yes, of course, they are working on different features.

I have been using this solution for many years.

We have not faced any concerns with respect to the stability of the tool; at least we have not seen any major issues with the tool where it malfunctions or anything, never.

It is a scalable solution. The beauty of TestComplete is that it's not tied to one scripting language. All the different scripting languages come with different abilities. You have different types of metrics and controls available with different scripting languages. That way we can drive our own framework. You can create many custom frameworks using TestComplete that will suit your organization. That's what we have been doing with many companies — it's an all-in-one solution.

The technical support is excellent. We don't deal with them directly, but the feedback that I have received from different clients is that it's really excellent. They are always attentive —  That's a feeling I've always got from different clients.

The initial setup is pretty straightforward. There's a good license server map for all those things — it's a fairly straightforward solution.

The licensing costs are in the range of $1,000 to $3,000.

We do a lot of consulting and training services for SmartBear. We try to educate the users in terms of the new features available in TestComplete so that they can do some smart automation. It's not just for automating some scenarios, you can optimize a lot of your effort.

On a scale from one to ten, I would give this solution a rating of nine.

Object-based recording, and data-driven testing.

Separation of data into Excel files made tests modifiable by QA personnel with limited development experience, and object-based recording kept maintenance to a minimum.

TestComplete's environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

This allowed us to schedule execution of lengthy tests for nighttime and non-core hours, and to synchronize tests with Jenkins build releases.

This freed up QA assets to perform more specialized testing and reduced redundancy.

Native test result reporting does not provide overview reporting methods for tests that span multiple project suites. Features that allow for flagging a test as dependent on the result of another in multi-project hierarchies while maintaining name-mapping segregation would be much esteemed.

I've used it for approximately eight months.

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

The initial set-up was exceedingly simple. There is a silent mode option available during set-up, which is very convenient when deploying to multiple machines or remotely.

We implemented it through an in-house team.

We did not maximize our ROI until we put somebody full-time on our TestComplete endeavours. The tool does have a learning curve, and it wasn't until we had an in-house expert on it that we began to see the benefits of automated testing over traditional QA roles.

The licensing options for TestComplete both running a licensing server. This prevents users from running more sessions simultaneously than purchased keys. This can prove problematic if you want multiple developers writing or running tests at once, and prevents you from using your key while a distributed test is running.

For pricing, carefully consider how many machines you want running the software, rather than the number of developers.

We also evaluated another SmartBear product called SoapUI. The change to TestComplete occurred because we changed our target from web applications to desktop.

My advice in regards to implementation would be to choose carefully which tests to automate, specifically focusing on lengthy procedures, tasks that require looping, or places where you want to test against multiple data sets.

Additionally, I found it beneficial to prefix my keyword tests with a character and number to provide logical ordering instead of alphabetic.

I also found it beneficial to record "undo" steps with each keyword test; this allows each test to be more stand-alone and prevents your test from being dependent on the state the previous test left the application in.

Finally, I would suggesting limiting the number of test applications per test suite to prevent bloated name-mapping schemes.

TestComplete is used for testing Windows desktop products; specifically the Embarcadero VCL interfaces created by Delphi/C++. All debug flags must be enabled during compilation that generates an output file required by TestComplete to identify and interact with the UI objects in the application.

An important feature of TestComplete is the ability to modularize testing. A lot of effort has been put into breaking the test script into reusable functions/methods that can be called from any test. A number of function libraries were created. This enabled reuse of code and kept the projects and project suites small. This is important because the size of these test artifacts have an impact on project loading time.

Test automation with TestComplete significantly (estimated 80%) reduced the regression timeline for a complex AUT with a large number of test cases. Automation with TestComplete has significantly shortened the feedback loop and the timeline to get a release production ready. A secondary benefit is that manual testers have begun thinking more technically about writing tests cases.

There are two major areas for improvement:

1. Version control integration embeds information in the project suite, and project files that include a direct reference to the location of the project or project suite in Team Foundation Server. When branching a set of scripts for the next version of the AUT, TFS gets confused about where the file should be mapped into source control. A workaround is to replace the hardcoded paths with relative paths.
2. There was also an issue object mapping related to the need to modularize the test script code. With the implementation of other automation test suites such at HP UFT, it is possible to have one object map for each function library. TestComplete has a limitation of only one object map per project. In order to modularize the code it was necessary to have a single shared object map used across multiple projects and project suites.

I have used TestComplete for 20 months. It was chosen as an incumbent toolset could not interact with the product to be tested.

Integration with third-party products; specifically Microsoft Team Foundation Server and HP ALM could not be overcome. A custom integration to HP ALM was written using the HP ALM OTA API.

TestComplete has been a stable product.

Customer Service can be slow to respond to electronic forms of communication and they do not have a way for a customer to speak directly to customer support. You create a ticket online, and request a phone call.

The team seems to be very knowledgeable when communication is established.

TestComplete was added to the toolset. HP UFT is used for automated tests for other products.

Setup is straightforward unless third-party tool integration is required. Integration with Microsoft Team Foundation Server is a little complex for initial configuration. Once it is understood the process is repeatable.

Implemented in-house. Implementation is not difficult to implement or write tests especially if you have experience with other test automation tools.

We have not done a ROI calculation. However, automated testing with TestComplete has cut regression test time by months.

HP UFT was tried but the object recognition did not work with the implementation technology.

We mainly use the solution for test automation. 

The programming capability, as well as the record and play, make the solution very easy to use.

The way that it's licensed is also quite attractive. 

The initial setup is pretty straightforward and simple. 

It's scalable. 

It has a very good run time.

The solution is mainly stable. 

The integration tools could be better. It would be useful if we could use it with other test management tools.

We'd like to see the solution add a few more features to the offering.

I've used the solution for more than five years. 

By and large, it is a stable product. There may be some bug fixes needed here or there. however, for the most part, it's fine. 

The solution is scalable. 

We aren't a very big company. Maybe 20 people are using the solution. Most of them are test automation engineers.

Technical support is average. It's not bad, nor is it outstanding. 

Neutral

I did not previously use a different solution. 

The company does, however, also use IBM tools. 

It is an easy solution to set up. The deployment takes minutes. It's very fast to set up.

We only need one person to maintain the solution. 

Our own team handled the implementation. We didn't need any outside assistance from integrators or consultants. 

The solution is around $1500. Some are perpetual licenses, and some get a yearly report card. 

Our team occasionally does test other options. 

We had some sort of partnership with the product.

I'm not sure which version we're using. The version we're using is likely two years old. 

I'd rate the solution seven out of ten. 

The name mapping function to manage the objects across different web pages and the support for different web browsers are the most valuable features for us.

We use TestComplete to do the automation test on our product’s web UI. In total, over 500 test cases are written and the automation coverage is over 80%. This helps to reduce manual effort dramatically, and by using the same script, we can run the test on both IE and Google Chrome.

The scripts are saved as binary files on disk. This makes it troublesome to perform version control and merge it with the work that is done by other team members. I am not sure what the reason is that SmartBear chose to save the files as binary, but if they can change to plain text for the scripts in the future, that would be helpful.

I've used it for one year.

Some of the objects/parent objects are dynamically created in the web page. During the test, TestComplete will have problem to identify these kind of objects. This will cause a lot of false rejection and block the following test cases. Thus it requires some rework when running all the test cases as a batch.

I have not used any customer service/tech support yet. I cannot provide more comments on that.

TestComplete is the first tool that I use for automation on UI and I have not used any other tools yet.

It is pretty straightforward. Just by following the instructions given in the document and do the corresponding configuration on the web browsers that you want to test. However, we also encounter some problems such as the Chrome blocking users from proceeding because of an unsafe SSL connection, but we found a work around eventually.

We implement the automation by ourselves, not through any other vendor team. My suggestion is that you need to decide how to divide the automation work into corresponding components carefully. Otherwise, it will be hard for you to share the common scripts with other team members. Also, the code architecture design is very important as well.

I am not responsible with the licensing quotation/renewal, but the benefit that TestComplete brings to us is, apparently, acknowledged.

I found the Keyword Test feature very valuable. Even with my limited programming knowledge, I was able to create automation tests using the Keyword Test feature. Also, the ability to create image based tests was extremely helpful, especially when automating tests for mobile devices.

TestComplete has enabled us to automate a lot of our tests for Android devices.

I've used it for three months.

I would rate it 10/10. Every time I had a question or issue I couldn’t figure out, they were right there to help.

Set-up was pretty straightforward.

We implemented it through an in-house team.