Oracle Application Testing Suite Reviews

Automation with recorded scripting: This is a nice feature, which allows customized testing scenarios to be created and edited depending on the testing needs at the time.

It has allowed more thorough testing with performance-based testing, in addition to regression testing and regular testing with each software release.

It does not provide easy diagnostics to ascertain end to end transaction monitoring. It shows statistics on individual class calls and it is challenging to aggregate the data into a single end-to-end transaction based on a single individual user experience.

We have used this solution for over two years.

No issues with deployment.

No stability issues encountered.

A system integrator installed the tool and has provided the support and expertise to initially run the testing and train our staff.

The tool is expensive to purchase and use based on licensing requirements for number of users.

Openscript and Oracle Load Testing (OLT) as I worked on performance testing with this tool.

It helped in running performance testing cycles and identifying the bottlenecks of the application, helping our clients to run their application successfully and smoothly.

Also, I used it for automating manual flows, which helped in saving time and effort in manual testing.

Improvements can be made in a number of protocol support areas, including enhanced Citrix support.

Also, the reporting utility for load testing could be improved.

I've used it for three years.

Garbage collection of our WebLogic server was not working properly when running large volumes of performance tests.

My experiences are good. There is a forum where you can post your issues and experts and others users will answer them.

Also, a service request which will be resolved quickly would be nice. There were a few patches given by the dev team to resolve issues.

Initial setup is straightforward. It will install Oracle Database, WebLogic Server, and other components by itself. At every step you will be prompted on your screen, but no other setups steps are required.

We implemented it in-house, but only a person who has good experience should be involved in the implementation, otherwise some issues might come later.

If you are using the Oracle bundled application then it is he best tool available in the market. But in some areas it lacks features and protocol support that its competitors, such as HP LoadRunner, have.

You should have a good and experienced person working with it, and also they should take people who have prior Java programming knowledge. The tool uses Eclipse IDE and Java script.

• Recording and playback
• Load testing 

Our benchmark load tests for a major release involving various scenarios with a combination of different virtual users relied on the recording, playback and eventual load testing capabilities of OATS.

1. The tool runs out of memory when the recording flows are long and involve many screens despite increasing the memory settings for OpenScript 
2. We found the tool impossible to use for client side performance testing without filtering out JS, CSS, PNG etc as the method would error due to a Java method size limitation. 
3. Parameterization of variables using regular expressions was not a foolproof way of running tests as it would mostly work for a single user but fail during load testing. 
4. Parameterization involving SQL queries was not straightforward 
5. Except a YouTube video/PDF which covers some basics and is probably outdated now, there are no tutorials available for other functionalities that the recording tool provides.There is no in-depth tutorial of the load testing interface and troubleshooting steps. 
6. Collection of system metrics - although there are multiple parameters defined, it did provide a holistic picture of the state of the system.We had to implement our own way of obtaining this. 
7. Load testing web interface did not provide details about errors. The only way to correlate was to inspect server logs.

One year.

Yes, with 12.2.x

Yes, 12.2 was not stable. Upgrading to 12.4.x resolved a lot of previously known issues

No issues encountered.

This tool did a much better job recording the http requests during web load testing than other open source tools. Cookie management and other url parameters were handled efficiently and were easier to identify in the tree view.

It was straightforward for the most part except that at times the services wouldn't run and un-installation messed things up.

In-house.

I believe we made a wise choice despite some shortcomings and hiccups.

We evaluated other options.

Read the tutorial and try out a few scenarios before going full throttle.

Importing a RUEI log into OATS to convert it into a load test

For a few years, OATS was the only tool that allowed us to load test our Computer Telephone Integration (CTI) with Siebel.

Our team is a contractor for a Siebel call center solution for the US government. Using Seibel means we are mostly an Oracle shop, but most of our test tools are HP. The one exception to that is OATS, which was formerly owned by Empirix when it was called eTester and eLoad. OATS has some promising features that frankly we wish we could utilize, but so far we haven’t had much luck doing so. Previously we did use it as a way of load testing the Computer Telephone Integration (CTI) aspect of Siebel. Even this needed to have custom Java code to implement though, and now that LoadRunner 11.5x is released, it can support our CTI load testing efforts as well, so we’ve gone away from using OATS for this.

2 years off and on.

Nothing significant.

No

Not up to 300 vusers, though this is not a large amount.

We use Loadrunner for all other load testing, but used OATS and Open Script just for CTI load testing.

Straightforward for initial setup, but complex for what we wanted to do. OATS runs as a suite hosted on a web server, so users access the test management piece via a URL. Scripting is done via Oracle’s OpenScript, which is their scripting development environment with Java, so being able to program in Java is probably the most important skill to have for using the tool.

Directly from Oracle.

We researched if this could be done in Loadrunner first.

Be sure your team has the right amount of expertise in OpenScript (Java) before deciding to use this product.

We use the cloud deployment model for this solution. 

We like that we don't need a separate management tool. This is a good feature. It also has an inbuilt performance tool which is on Flash. It has very good record and playback features as well. The inspection tool is also very good. Overall, since it comes with all the three packages, it's very good.

It has a very good inbuilt reporting system.

There is no support for this particular tool and you don't have a very big knowledge base on Google. It should order the tests so that you can automatically write a path and test it on it. The object identification feature can be enabled and leveraged to other levels and there's some kind of knowledge base. These are the two main areas that need improvement.

The pathfinding at times is slow when we are using it. The tool's performance can be improved. 

The third thing, is the help and the support or the knowledge base needs improvement. Otherwise, it's overall a very good tool.

It should have proper integration with CI/CD tools or any plugins. There's good documentation with Jenkins or JIRA.

It should be a little more flexible like Selenium for the user to try with different locators or the different writing of XPath. The slowness can be identified and when we try to automate any internet, the application which can only be opened in Internet Explorer has some challenges.

It needs better automation. I think it is there but some enhancements should be done. Overall, it is a very good tool that is better than some other automation tools on the market. It supports various entrepreneur platform applications and applications developed with different coding background and it's all very good.

I have been using this solution for around four years.

It's a stable one except for one or two incidents, it's stable. 

It has good scalability. It can integrate it with CI/CD tools and all those things. There is no inbuilt configuration or setups available to link it with other tools or scale it with other CI/CD tools. Those tools are lacking configuration abilities. It's a lack of knowledge from our side of the this because of the unavailability of the knowledge base. We haven't tried with different stops.

We have around seven or eight users using this solution. We require two staff members for the deployment and maintenance. 

I would rate their technical support 2.5 out of ten. 

I have only used Oracle. I didn't use a previous solution. 

It's really a very good tool. My advice to somebody considering this solution is to speak to the vendor, make sure to make your requirements clear and have clear answers from the vendor's side. Ask what support is available and what the scalability terms are and what features are available to integrate with CI/CD tools.

The tool is very good. There's no doubt about it so. In terms of the properties or object identification media, there could be some more or better documentation, a better knowledge base or better enhancements should be available.

I would rate it a seven out of ten. 

1. Java as native supported language.
2. User friendly UI / Tree view to work with adding steps.
3. The initial kick of time of a functional test automation script is lesser than other popular tools like QTP. But not as quick as selenium :). But selenium supports of automation of only web applications.
4. Stable enough for Oracle EBS Applications and other products of Oracle, there are some components which are not recognized by other test automation products but are recognized by OATS ( Oracle Application Testing Suite )

To provide test automation support for other products like SAP, Windows and Java Applications when it comes to Functional Test Automation testing. Which otherwise they have a pretty good coverage to test all Cloud or Web Based applications.

I have used for about 8 years now, had worked with this tool from the time of inception at oracle, as I had worked at ORACLE for 7.5 years and used OATS for most of the time. It was our default solution for test automation of Oracle EBS Applications.

OATS 12.5.0.2 has a pretty smooth installation process.

In budding stages, the product had serious stability issues, but ORACLE has put its best people behind and made the product quite stable. Now I feel its the best tool to automate any of the Oracle Products, which include Oracle EBS Applications, PeopleSoft, JD Edwards, Siebel, Adobe Flex, ADF applications, etc..

The technical support on Linkedin Groups and Direct support are good, would rate it as 4 out of 5

We used Winrunner earlier and later HP had stopped its support so we had to switch to another Test automation product and when an internal tool could support, there was no point in paying to external companies, while I was working with ORACLE

Simple and easy

There was huge ROI, as we stopped all Licenses that were paid to HP and used something internally at ORACLE

ORACLE is giving at a very competitive rates to all its customers, and its a simple licensing process.

Yes, we tried with QTP, but it was doing no good than internal tool for test automation of Oracle EBS Applications.

It's a really amazing solution, which I have ever used, three tools which I used most were Winrunner, QTP and OATS. I feel OATS is best, even now I am out of ORACLE I believe OATS has much more value, compared to others. and I liked it so much that I started writing my experiences over my personal blog which is http://testingtools.co

Test Manager “TESTS”

1. I can organize test scripts pretty well.

2. The custom dashboard views are nice to look at.

3. Sorting tests is pretty easy.

1. The installation routine desperately needs improvement. It was very easy to mess up.

2. The “un-install” is basically not supported and resulted also in a messed up computer.

3. In the OATS Administrator tool, setting user assignments and roles to projects was tricky, scary and dangerous. Updating somebody’s account access sometimes had unexpected and bad results like getting “locked out” of the product.

Two years.

The test installation pretty much destroyed my computer’s laptop performance. It had to be “reimaged” to recover. It was eventually installed on a dedicated server.

Some user accounts had to be removed and new ones created because of corruption.

For example:

1. At some point my user account could not move or organize the position of folders and tests.
2. Another user would crash the entire product whenever she choose to open one specific project.

I am not aware of any issues.

Customer Service:

The training people were good.

Technical Support:

I have not had to deal with support.

We also use TFS. We are using OATS because my company purchased an Oracle ERP solution.

There did not appear to be a simple installation routine. The initial setup was complicated with many parts to consider including binaries and a local database.`

I believe a vendor did the original implementation.

Brace yourself for bumps in the road. Have patience. Find talented people to help.

The best feature of this and Load Test is simplicity. They are probably not as functional as the corresponding HP products, but you can start using them immediately after installation. Script development platform is based on Eclipse IDE and was much better and stable than LoadRunner VUGen 11.5 and earlier versions.

Load Test has perfect monitoring features. It fully supports JMX (we were able to monitor WebLogic internals in a couple of clicks), modern SNMP versions, flexible custom metrics, etc. We didn’t have any problems with collecting data we wanted from Windows, Linux, Solaris machines, and that’s really great. I can’t compare it with HP SiteScope (don’t have much experience with it), but it definitely beats standalone HP LoadRunner facilities.

I also liked the web interface of the Load Test controller. It supports remote test execution and provides quite convenient API. You don’t have to use separate solution as Performance Center or LoadRunner Analysis to create test reports and provide remote access to your test artifacts.

The scripts are written in Java, so migration from C (used in our LoadRunner scripts) to Java significantly accelerated and simplified script development. We have also managed to monitor our infrastructure more precisely.

Load Test provides great facilities and templates to monitor other Oracle products and create simpler scripts to automate load test pipeline. You load the script into Load Test, run the test, and generate the report.

I’m not sure about the functional testing API, but the load testing one wasn’t rich. We had to extend it by adding lots of functions, but it can be easily done. Transaction handling was very simple and I’d say quite primitive. We were unable to pause, dynamically create them and change their state, such as transaction name and other properties.

API was convenient but too poor, but I know that Oracle has extended it. Reporting features were very simple as well, especially report templates.

We've used it for about a year. We’ve migrated our scripts and related test automation (e.g. report generation) successfully. It wasn’t perfect. We had to migrate our HP LoadRunner 11.5 scripts to Oracle Linux Test, and we had to do a lot of manual work to add features missing in it, such as writing custom functions in scripts, calculate target metrics with SQL on its database, etc. Though it was mostly stable and reliable, Test Management wasn’t. However, things may have changed in updated versions.

As far as I remember, we had lots of issues with Test Management stability, but the recent version are stable. We had several issues with Load Test monitoring, but were quickly fixed by Load Test developers.

It's excellent. We usually got help in a couple of days (hours in critical situations). The Load Test team frequently asked for feedback from our managers and included our suggestions into releases.

We tried a lot of solutions and used HP LoadRunner for six to seven years before switching. When Oracle Application Test Suite became ready for enterprise usage and met our needs, we started to use it, partially because it’s free for usage with other Oracle products.

It was really simple. The installation package can do everything for you (install WebLogic and Oracle XE) or you can customize the process. Anyway installation process is very transparent and flexible.

We did it ourselves with a little assistance from the development team. The documents are a good starting point so almost all of the questions can be addressed after reading the manual.

It was completely free for us, because we use it with other Oracle products.

It is a simple yet useful tool for performance testing. Compared with LoadRunner, it’s definitely less functional. I’d recommend you to try their free (development) version to understand what it does and does not.