Oracle Application Testing Suite Reviews

Robustness of Oracle Application Testing Suite to manage end-to-end testing of projects, especially in Oracle ERP implementation projects.

Oracle Application Testing Suite is very good for managing testing through Oracle Test Manager (OTM), automating test cases with Oracle Functional Test (OFT) and conducting performance testing in Oracle Load Test (OLT).

Implementation of Oracle Application Testing Suite package has increased the overall efficiency. Test cases and defects are now tracked well in Oracle Test Manager which help management to have better visibility and traceability. Increasing automation coverage has been reducing manual efforts in executing test cases and in a recent test cycle automated execution, it saved seven days of manual execution effort in a two-week test cycle.

OTM Test Report – We needed to develop custom reports as there is less flexibility.

OFT Support for Non Oracle ERP/CRM Product – As of now OFT supports only Oracle ERP/CRM Products. OFT has to support other vendor’s ERP/CRM packages to increase its base.

I've used it for over five years.

In Windows 7 and in Windows 2008 R2 server deployment is very straight forward. In Windows 2012 R2 server, deployment is a little bit tricky as .NET Framework 3.5 is not a part of OS distribution. One has to separately install it. You have to take care of this before going for installation in 2012 R2 Server. For Linux/Unix deployment you have to be well aware of Unix/Linux OS environment and you need database administration knowledge/support.

No issues encountered.

No issues encountered.

Customer Service:

It's good, 7/10.

Technical Support:

It's good, 7/10.

We switched because of lower licensing costs and it has good support for Oracle ERP/CRM.

It wasn't complex.

We implemented it in-house. Though the initial set up is not very complex, implementation should be done by resources who have good experience in Oracle Application Testing Suite and have a testing background. Database administration knowledge or support is also required.

I don't have specifics, but the product is cheaper than similar products from HP UFT and Quality Center.

Oracle Application Testing Suite is best suited for managing testing activities in implementation and upgrade project for Oracle’s ERP/CRM products.

• Recording and playback
• Load testing 

Our benchmark load tests for a major release involving various scenarios with a combination of different virtual users relied on the recording, playback and eventual load testing capabilities of OATS.

1. The tool runs out of memory when the recording flows are long and involve many screens despite increasing the memory settings for OpenScript 
2. We found the tool impossible to use for client side performance testing without filtering out JS, CSS, PNG etc as the method would error due to a Java method size limitation. 
3. Parameterization of variables using regular expressions was not a foolproof way of running tests as it would mostly work for a single user but fail during load testing. 
4. Parameterization involving SQL queries was not straightforward 
5. Except a YouTube video/PDF which covers some basics and is probably outdated now, there are no tutorials available for other functionalities that the recording tool provides.There is no in-depth tutorial of the load testing interface and troubleshooting steps. 
6. Collection of system metrics - although there are multiple parameters defined, it did provide a holistic picture of the state of the system.We had to implement our own way of obtaining this. 
7. Load testing web interface did not provide details about errors. The only way to correlate was to inspect server logs.

One year.

Yes, with 12.2.x

Yes, 12.2 was not stable. Upgrading to 12.4.x resolved a lot of previously known issues

No issues encountered.

This tool did a much better job recording the http requests during web load testing than other open source tools. Cookie management and other url parameters were handled efficiently and were easier to identify in the tree view.

It was straightforward for the most part except that at times the services wouldn't run and un-installation messed things up.

In-house.

I believe we made a wise choice despite some shortcomings and hiccups.

We evaluated other options.

Read the tutorial and try out a few scenarios before going full throttle.

We use the cloud deployment model for this solution. 

We like that we don't need a separate management tool. This is a good feature. It also has an inbuilt performance tool which is on Flash. It has very good record and playback features as well. The inspection tool is also very good. Overall, since it comes with all the three packages, it's very good.

It has a very good inbuilt reporting system.

There is no support for this particular tool and you don't have a very big knowledge base on Google. It should order the tests so that you can automatically write a path and test it on it. The object identification feature can be enabled and leveraged to other levels and there's some kind of knowledge base. These are the two main areas that need improvement.

The pathfinding at times is slow when we are using it. The tool's performance can be improved. 

The third thing, is the help and the support or the knowledge base needs improvement. Otherwise, it's overall a very good tool.

It should have proper integration with CI/CD tools or any plugins. There's good documentation with Jenkins or JIRA.

It should be a little more flexible like Selenium for the user to try with different locators or the different writing of XPath. The slowness can be identified and when we try to automate any internet, the application which can only be opened in Internet Explorer has some challenges.

It needs better automation. I think it is there but some enhancements should be done. Overall, it is a very good tool that is better than some other automation tools on the market. It supports various entrepreneur platform applications and applications developed with different coding background and it's all very good.

I have been using this solution for around four years.

It's a stable one except for one or two incidents, it's stable. 

It has good scalability. It can integrate it with CI/CD tools and all those things. There is no inbuilt configuration or setups available to link it with other tools or scale it with other CI/CD tools. Those tools are lacking configuration abilities. It's a lack of knowledge from our side of the this because of the unavailability of the knowledge base. We haven't tried with different stops.

We have around seven or eight users using this solution. We require two staff members for the deployment and maintenance. 

I would rate their technical support 2.5 out of ten. 

I have only used Oracle. I didn't use a previous solution. 

It's really a very good tool. My advice to somebody considering this solution is to speak to the vendor, make sure to make your requirements clear and have clear answers from the vendor's side. Ask what support is available and what the scalability terms are and what features are available to integrate with CI/CD tools.

The tool is very good. There's no doubt about it so. In terms of the properties or object identification media, there could be some more or better documentation, a better knowledge base or better enhancements should be available.

I would rate it a seven out of ten. 

One feature that I find valuable and very useful is the ability to use a different version of the database for the repository other than the Oracle Lite DB that it comes with by default. This means you can make your database repository fully redundant and infinitely scalable.

OATS provides the functionality to institute a corporate-wide standardized method and approach for all types of application testing. This tool is perfect for organizations that do a lot of in-house development and/or regularly perform maintenance. I would say it is a mandatory tool to have and use if your organization is planning to perform a major upgrade or begin a large implementation.

There is one piece of functionality that is currently not offered and would be very good to have: Support for Digital Certificates authentication and Single Sign-On (SSO). This method of authentication, to the best of my knowledge, is still not supported by OATS and would be a great addition to the tool.

I used OATS for over five years. While working for Oracle Consulting, I've seen the tool used successfully at numerous clients.

It is easy to deploy.

It is very stable.

It is very scalable.

Within the Oracle ecosystem, OATS has a small but very capable and technically advanced support team. To the best of my knowledge, most of these folks came to Oracle as part of the acquisition process of OATS, meaning they were supporting and/or working in product development of OATS for a long time.

I used HP LoadRunner in the past and do not really have a preference for either OATS or LoadRunner.

The initial set-up/install/deployment is very straightforward and should not present any problems for any reasonably technical person.

While working with Oracle Consulting, I implemented OATS myself personally as a hired Oracle consultant as well as together with clients' in-house teams.

One advice I would give if you are planning to do a multi-node install on Linux/Unix is as follows: Unless your organization is very familiar with the product and complex architectures that require fault tolerance have someone from Oracle Consulting come in for a short engagement, set everything up, document everything and do a knowledge transfer. This way you will be ahead of the curve from the start. From personal experience, any kind of rework is always more costly than doing it right from the start.

It's a little complicated to calculate an ROI of a product that isn't replacing another product. I would say the best way to calculate an ROI is not to focus on OATS and its associated cost, but to calculate the amount rework needed to fix issues if OATS were not to be used.

There are basically only two products that cover the space of Performance/Scalability/Functional Testing on the market. OATS from Oracle and LoadRunner from HP.

One advice for anyone planning to implement OATS: Make sure there is an existing Java skill-set in your organization as OATS' recording language is Java.

Test Manager “TESTS”

1. I can organize test scripts pretty well.

2. The custom dashboard views are nice to look at.

3. Sorting tests is pretty easy.

1. The installation routine desperately needs improvement. It was very easy to mess up.

2. The “un-install” is basically not supported and resulted also in a messed up computer.

3. In the OATS Administrator tool, setting user assignments and roles to projects was tricky, scary and dangerous. Updating somebody’s account access sometimes had unexpected and bad results like getting “locked out” of the product.

Two years.

The test installation pretty much destroyed my computer’s laptop performance. It had to be “reimaged” to recover. It was eventually installed on a dedicated server.

Some user accounts had to be removed and new ones created because of corruption.

For example:

1. At some point my user account could not move or organize the position of folders and tests.
2. Another user would crash the entire product whenever she choose to open one specific project.

I am not aware of any issues.

Customer Service:

The training people were good.

Technical Support:

I have not had to deal with support.

We also use TFS. We are using OATS because my company purchased an Oracle ERP solution.

There did not appear to be a simple installation routine. The initial setup was complicated with many parts to consider including binaries and a local database.`

I believe a vendor did the original implementation.

Brace yourself for bumps in the road. Have patience. Find talented people to help.

Importing a RUEI log into OATS to convert it into a load test

For a few years, OATS was the only tool that allowed us to load test our Computer Telephone Integration (CTI) with Siebel.

Our team is a contractor for a Siebel call center solution for the US government. Using Seibel means we are mostly an Oracle shop, but most of our test tools are HP. The one exception to that is OATS, which was formerly owned by Empirix when it was called eTester and eLoad. OATS has some promising features that frankly we wish we could utilize, but so far we haven’t had much luck doing so. Previously we did use it as a way of load testing the Computer Telephone Integration (CTI) aspect of Siebel. Even this needed to have custom Java code to implement though, and now that LoadRunner 11.5x is released, it can support our CTI load testing efforts as well, so we’ve gone away from using OATS for this.

2 years off and on.

Nothing significant.

No

Not up to 300 vusers, though this is not a large amount.

We use Loadrunner for all other load testing, but used OATS and Open Script just for CTI load testing.

Straightforward for initial setup, but complex for what we wanted to do. OATS runs as a suite hosted on a web server, so users access the test management piece via a URL. Scripting is done via Oracle’s OpenScript, which is their scripting development environment with Java, so being able to program in Java is probably the most important skill to have for using the tool.

Directly from Oracle.

We researched if this could be done in Loadrunner first.

Be sure your team has the right amount of expertise in OpenScript (Java) before deciding to use this product.

Automation with recorded scripting: This is a nice feature, which allows customized testing scenarios to be created and edited depending on the testing needs at the time.

It has allowed more thorough testing with performance-based testing, in addition to regression testing and regular testing with each software release.

It does not provide easy diagnostics to ascertain end to end transaction monitoring. It shows statistics on individual class calls and it is challenging to aggregate the data into a single end-to-end transaction based on a single individual user experience.

We have used this solution for over two years.

No issues with deployment.

No stability issues encountered.

A system integrator installed the tool and has provided the support and expertise to initially run the testing and train our staff.

The tool is expensive to purchase and use based on licensing requirements for number of users.

Oracle Application Testing Suite is used for automating.

Oracle Application Testing Suite's most valuable feature is it works very smoothly with all Oracle Java-based applications.

Oracle Application Testing Suite could improve by offering desktop-based application automation. It is lacking in this area at the moment.

I have been using Oracle Application Testing Suite for approximately seven and a half years. 

Oracle Application Testing Suite has been used for many years and it is highly stable.

I rate the stability of Oracle Application Testing Suite a nine out of ten.

Oracle Application Testing Suite is easy to scale. We can use multiple applications and use different scripts. The solution is highly scalable.

We have approximately 15 people using this solution in my organization.

We have raised tickets with Oracle for support. The support is the best. However, depending on the level of support that you have. If you have the 24/hour seven days week support, you will not have any issues with the support.

I rate the support from Oracle Application Testing Suite a 

We have other solutions similar to Oracle Application Testing Suite, such as Automation Anywhere and UiPath. We found that Oracle Application Testing Suite is the best for form-based automation.

The initial setup of the Oracle Application Testing Suite is simple.

The price of the Oracle Application Testing Suite is not expensive. It is less expensive than other solutions.

My advice to others is if they are using any Oracle Cloud Fusion-based or on-premise-based application, this is a good solution to use.

I rate Oracle Application Testing Suite an eight out of ten.

On-premises