What is most valuable?
VBScript is easier to learn than Java.
There are many new and old features in UFT 14.01 which are valuable.
In UFT 14.01, new support has been added for AngularJS 2.0 and 4.0. While most of the AngularJS objects were recognized out-of-the-box in UFT 12, 14.01 now also recognizes grid objects, such as Web tables.
Hidden among the kitchen sink of features is a new Data Generation tool called the Test Combinations Generator.
- Need 10 to 100 random city names?
- A range of random dates?
- What about a list of first and last names? Random numbers, emails and passwords? Even IP and MAC addresses?
- Need a list of random part numbers in a custom format? The Regular Expression data generator will fill your need.
This data can be exported to internal Excel data worksheets that give upfront visual access to data for starting data driven projects.
The UFT has multiple approaches for object identification including:
- Object Repository
- Descriptive programming
- Object collections
- Relational recognition (Below or right of another object)
- Insight objects (Image recognition)
- Virtual objects
Ensures you can find objects by the DOM, CSS and XPath, but descriptive programming is often easier to read.
Connections to MS Excel and popular databases allow users to move to the advanced frameworks of data acquisition with SQL queries.
The fully-redesigned Run Results module is simply beautiful. Someone tried to compliment me on rolling my own results page, and I admitted what they were looking at was completely out-of-the-box.
Let’s not forget being able to make external calls to supporting tools, like AutoIt, as well as the Windows API, to provide expanded functionality.
Lastly, support of version control in both the stand-alone tool and the ALM repository.
How has it helped my organization?
Our current project features more than a 1000 manual test cases, which took several days and resources to execute. Now, the suite executes in six hours and less than two when run on multiple machines.
With Jenkins connected, or the new cloud-based StormRunner Functional Testing, the tests can be launched anytime, even at the end of day, and be ready for analysis the next morning.
What needs improvement?
It is important to note here that another Micro Focus product, ALM (aka Quality Center and Test Director), has long suffered from “click-itis” since its release. Nearly every dropdown is collapsed by default. Two option choices are implemented with dropdown lists rather than radio buttons. Most edit fields require opening an additional edit panel in the unlikely chance you need to bold or italicize a step summary. So navigation is a perpetual repetitive click-expand experience that quickly becomes a total turn off to the user.
I mention this shortcoming of ALM because, unfortunately, this design is seeping into UFT as well. UFT 14.01 has eliminated the "Run" button from the shortcut menu bar when a function library window has the focus. In effect, where you could click just one button to run a script in UFT 12.54, you will now have to click the Test tab first to get the Run button to appear to be clicked. Fortunately, the F5 (Run) and F11 (Step-through) keyboard shortcuts still work regardless of the window with focus.
It looks like User Acceptance Testing of the product is getting bypassed entirely because this design has precedence in UFT. Throwing an object to the Watch window almost always requires two attempts. So, train yourself to always click the Watch window tab first. More “Click-itis” for the user.
UFT 14.01 did make one long awaited improvement that is very welcome. The object Spy utility can now be left open while writing code. This means you can paste multiple property names and values from the Spy into your object identification code without having to repeatedly close and reopen the Spy tool. Now, if two Spy windows could be launched to compare the properties of two similar objects, that would be another welcome click-reducing feature.
The list of good features far exceed the bad. Here are a few that could be addressed in upcoming releases to get the tool to a perfect rating:
- Double-clicking a function in the Toolbox window used to take the developer to the source code. Now, it throws a function call wherever the cursor happens to be. This runs the risk of breaking code. UFT 14.01 fixes a feature that would replace selected code. It now just jams the code in the middle of the selection, which is still not great.
- Jumping to functions is supported from any Action/BPT Component code, but not from inside a function library where the target function exists in another library file. Workaround: Select search entire project for the function.
- The Run results module counts the number of Fails and Warnings in a test, but not the total Pass results. Workaround: Roll your own results counter code.
- No RegEx support of integers properties in the Repository and Descriptive Programming. If the tool just allowed [1-9]\d+ in the Height and Width properties, the returned object collections would exclude all non-visible objects. Workaround: All objects in an object collection need to be tested for height or width to ensure visibility, but slowing execution.
For how long have I used the solution?
I have worked with QTP/UFT for 13 years continuously. My projects over the years have included the tool along with the use of ALM (aka Quality Center or Test Director), Business Process Testing (BPT), and TAO for SAP.
What do I think about the stability of the solution?
The product is extremely stable. It's been my tool of choice for nearly two decades because it is solid.
What do I think about the scalability of the solution?
Scalabiliy is entirely up to the framework design. While Record and Playback are available for new users, it will result in fragile high maintenance and unscalable test suites. That's true of most automation tools, so a hybrid framework design approach is always highly recommended. Fortunately, UFT is extremely flexible in design.
Advanced developers can go so far as to design a framework which translates to Plain English ("Click Ok Button") into script code (WebButton("InnerText:=Ok").click). This leads to function designs similar to Gherkin & Cucumber, bypassing the object repository entirely.
With BPT (Business Process Testing), non-technical users can easily build test cases inside of ALM (Application Lifecycle Management) from scripted components designed by automation engineers. Simply copy an existing test and add additional components that meet your test requirement.
Scaling to push multiple tests executions with different data sets is also easy to implement particularly with the new Data Generation Tool described above.
Lastly, scaling object class methods to add new functionality or modify the process of execution is achievable with just a few lines of code. The tool even provides a Function Definition Generator Wizard to help build the code the first time.
How are customer service and technical support?
Micro Focus support is OK. Orasi support is outstanding.
Which solution did I use previously and why did I switch?
Prior to using VBScript-based UFT/QTP, I used Mercury Interactive's C-based Winrunner before the product was discontinued.
How was the initial setup?
Initial setup is very straightforward and takes about 45 minutes, including one system restart. The tool installs the bare minimum of add-ins. To add more, it takes about 15 minutes.
Installing the Terminal Emulator add-in is easy, but configuring it is a bit tricky. This being a bit of a horse and cart issue, as you can't see the configuration option unless the Emulator is up and connected first.
What's my experience with pricing, setup cost, and licensing?
For the price of five automation licenses, you simply would not be able to hire five manual testers for two years worth of 24/7 manual testing work on demand.
While many will argue there are other tools available that are free, you may find it hard to find one which supports so many new and legacy web technologies, terminal emulators, and Windows thick client applications. It's the kitchen sink of tools with an easy to learn language, a solid history, and extensive support resources.
Which other solutions did I evaluate?
I only work QTP/UFT engagements, however I do compare functionality of other tools in my spare time, including Selenium, SmartBear TestComplete, and HPE LeanFT.
What other advice do I have?
AngularJS support, Data Generation and New Spy features are great, but creeping “Click-itis” sours user experience.
There are so many features! The tool is easy to learn, flexible, and extensive.
Be sure to have new automation engineers trained beyond basic YouTube videos, and avoid on the job training. This will prevent rookie mistakes that will generate unmaintainable scripts and re-work in the future.
Micro Focus provides tool training, as does Orasi and RTTS in New York.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Thanks Mark!