Worksoft Certify Reviews

The primary use case is more on SAP for automating all our manual business work, as well as doing regression testing. We do end-to-end regression testing on SAP.

The product has worked very well with SAP.

Worksoft is used for manual processes that needed to be automated. This has helped us make our processes be faster and more efficient because there are no manual interruptions or errors.

For my processes, Worksoft makes them faster when creating scripts.

Capture 2.0 is a very good feature, on which we can record very easily and get documentation generation and testing acceleration. 

The product had some UI issues. In the next release I heard the UI issues will be lifted up (version 11), I am excited about it because the product will have more UI features. We are thinking of upgrading our existing Worksoft Certify from 9 to 11, when it is released, as this will be good for the company and help all our users.

The stability is pretty good what I have used sofar with SAP

The scalability is good.

The technical support is pretty good. If you open a ticket, they are able to answer it within 24 hours. They are able to support us.

We previously did manually testing and used other tools. With Worksoft, we can more see the productivity and benefits that it provides. We can also do more testing, making easier for all the users. 

We used the Worksoft for the deployment. They came and helped us. We have a good relationship with them.

Worksoft Certify is an investment. We see value in it more than the money. We see value in it because the user can look at different aspects of testing and what they can do.

This solution has enabled us to automate in order to tremendously save time. Only first time when you are recording and creating the script will you spend some time with it, the rest of the time, you are just executing. If we do one manual process, it could take approximately two hours. The same process using Worksoft probably takes ten minutes.

It help cut test maintenance time too. An eight hour manually project using Worksoft can be done in an hour.

Worksoft is good for SAP to do your automation and testing.

We are not using it for web UI testing.

The primary use case is regression testing and test automation.

We use this solution for end-to-end testing of packaged applications, like SAP, ServiceNow, and some in-house web apps (which we built).

We have already found some defects using the test automation. 

One of the bigger value-adds that we had was extracting data from our warning systems to be inputted into our new learning system.

We automated because of the value. Our business users do a lot of regression testing. They often don't have the time when we need to get the testing done, so this frees them up to do more value-added work for the company.

• Speed, as far as getting tests completed on time. 
• Its ease of use and the value of that. It is a lot easier to maintain test scripts on Worksoft Certify than on other testing tools that we have had in the past.

We have had run ins with some bugs on Business Process Procedure (BPP) and Execution Manager. 

We also did upgrade Worksoft Certify recently to clear up some issues with server fogging.

More than five years.

It scales pretty well.

I would rate technical support as a seven out of ten. Sometimes, we end up going back and forth with Level 1 support. Usually, if we go back and forth, it is email most of the time, and sometimes it is once a day with 24-hour turnaround. We usually understand what they require and trying to give as much information as possible. We still need to tell them to kick it up to Level 2 support or at least have a meeting with them, so we can show them what is going on. Sometimes, it does get a little frustrating, but once we either have the meeting or get Level 2 support, the issue gets resolved pretty quickly.

We had and still use QuickTest Professional from Micro Focus a little bit, but we were approached back in 2008 about getting into Worksoft Certify.

We went with Worksoft Certify because of its ease of use. You don't have to know a scripting language, like with QuickTest Professional. Also, it has ease of maintenance.

The initial setup was not really complex, just a little. We did not understand the full setup; it was just confusing. We really didn't get a full explanation of how things connected together until we actually started to use it. Once, we start using it, things became much clearer.

During the initial deployment where we received execution suite, the necessary information was not really provided. However, it was a fast deployment. We had to meet some deadlines, so people just came in and did the installs, or helped us do the installs over the phone, then afterwards we realized how things were connected.

We used Worksoft for the deployment, who was very good.

We have seen ROI by being able to free up and give time back to the business for other value-added work.

In 2008, we evaluated SAP TAO. The person who came in to do the proof on concept wasn't able to get the test scenario working at all. He had three and a half days to do it, and he couldn't do it. When Worksoft came in, they had the same length of time and were able to get that one plus another one partially working in the same time frame.

Worksoft generally seems to want to make sure you are successful at what you are trying to do. I haven't come across an employee from Worksoft that isn't willing to help. A lot of times you do get that from other salespeople, and that is just not the case from what I have seen from Worksoft. 

The product has a lot of benefits as far as getting testing done. It gives you some value back.

The company is using it across our IT landscape on different projects. We use it to test web applications, SAP mainly. Right now, we're doing a big conversion project. The one that I am working on is a big conversion from an old mainframe system over to SAP. Thus, we are using it to automate all the UAT test cases.

We have also used it in other areas of the company for ServiceNow upgrades and general web design stuff.

For this SAP upgrade or implementation, the business users are creating building blocks in their areas, then they're changing the building blocks into long end-to-end scripts to do a complete end-to-end testing to speed up the UAT. It definitely has a bigger coverage of UAT testing.

• It is user-friendly. We can give the tool to business users, and they're able to use the tool pretty efficiently without a whole lot of training. 
• The reporting features are nice. 
• It is easy to take someone else's work and manipulate it to use yourself. 
  
• It is not heavily code-based, so you can pick it up and automate very quickly.

For Execution Manager, I would like it to be more robust interface and be able to view the remote machines full screen instead of a little window. This would be a great upgrade for us.

I would also like more customized reports without having to print out big reports.

It is a very stable product. When we have had upgrades, it's been pretty seamless. Older versions of our scripts work in the newest update without a lot of rework.

It is very scalable with users. We have business users who are scripting. We have Worksoft developers who do more complicated work. Then, it moves over to the people who do execution and process through Execution Manager, so we have several different layers of users doing different tasks.

The technical support is very good. They are very responsive. When we have created tickets with them, there is usually a 24-hour turnaround time, then we are contacted back. Their interface is good for back and forth communication.

It is not practical to do manual testing anymore due to the volume of it. The amount of data variations is too great to test manually. If we were going to do it manually, we definitely would not have full coverage, where now we can get pretty close to full coverage on our tests.

When you are implementing Worksoft Certify, it is probably best to bring some Worksoft consultants in to help get you setup, and set it properly. Because if you don't get your initial systems, like your folder structures and your naming conventions, set at the beginning, there is a lot of rework to get it to work. We did have to go through that.

We save probably 50 percent of our time. The tool does what it's supposed to do, and we are able to actually work.

Worksoft Certify is a good product. The customer support is really helpful and supportive. They are always upgrading their products to new features, which we like. It is a pretty stable tool, which doesn't require a lot of maintenance.

Our environment has SAP Fiori. They are also doing a HANA implementation. As far as the web, I don't work on that side as much.

I haven't used the Capture 2.0 very much.

We use Worksoft Certify for testing and non-testing. 

• Under the testing umbrella, we use it for regression testing, disaster recovery, validations, and system refresh testing. This has been our main focus. 
• Recently, we have started getting more requests for more RPA-light type of work, which is not testing. It is using the same skills, process knowledge, and tool sets to do work that would replace manual repetitive tasks with automation.

We use Worksoft Certify for some basic ServiceNow functionality or Workday releases, weekly and quarterly releases. For SAP Hybris and ECC, we are also using it for our internal security protocol testing. So, we test Office 365 and Windows 10 compatibility. We test some Excel functionality and file sharing, as part of our security protocols. The most in-depth end-to-end testing that we have is in SAP.

We have done a lot of manual testing. We still do a lot of manual testing for our projects. We've eliminated a significant amount of manual testing with our system refresh, automation, and for technical upgrades where changes are known. However, for projects with new enhancements and functionality, we are having a slower time penetrating into them. With regression testing, we have completely replaced it with automation. 

Now, we are trying to shift as much as we can to start automating processes earlier in the project lifecycle, but it has still been a bit of a challenge. This is one of our stretch goals for this next year. The non-testing area is where we have had the most growth over the last six months.

The freshest example is some of the RPA-light activities that we have done. We have been able to save on a lot of manual work for some very highly skilled, expensive resources. This has been able to free up a significant amount of their time so they can spend more time on innovation and more creative, value-add activities. That's been one of the more rewarding things that we've done, and the most appreciated. 

We run our tests and our full suites every week. We have them scheduled in batches so certain sections of our scripts run every day, then we run them through the whole suite every week. That is how we maintain them by running and repairing them.

To run them every week and make repairs takes us maybe two hours. Because they are scheduled on batches to run, it's lights out. They are pretty hardened at this point, so there are not a lot of repairs for data, etc. If there is a break, it is usually because there has been a change to a process that we were not aware of. Our automated scripts are the company's best business process documentation, as we don't have a business process management tool. Therefore, if anybody wants to know about our business processes, they come to us for something like training or new production support resources. They will come and watch our automation if they want to understand how the systems work.

The ability to run multiple processes at the same time remotely or on a schedule. So, we have some testing that we do every day, and it is pretty much lights out. It is unmanned. We have some virtual machines that run on a schedule. Therefore, it's out of mind testing unless there is an issue. They are very hardened tests. If there is an issue, it means that there is something that we needed to catch, so it is always a good catch. This has given us a lot of flexibility because now we can use those resources in other ways. Besides the basic automation capability, it has been great having the ability to test multiple applications and multiple processes at the same time and overnight, then just receive the results.

I have always appreciated the Capture tool. I'm excited about the new enhancements that have been made to it. I think this will make adoption a lot easier because the tool is a lot easier to use and has more capability. I'm excited, because this is a good time for us as we are expanding into more RPA-light space. It will be easier if we could have some of our more technical resources doing a lot of this capture work. Then if they do something wrong, it's very intuitive to stop, delete that step, and start over. It will make adoption a lot better, and we'll get better requirements and processes documented that we can then convert into automation.

The time it takes users to document, then for us to automate, depends. We have a lot of end-to-end business processes, but they have to go across functional teams. They tend to get passed around a bit. Worksoft definitely saves time. We were literally getting processes on napkins. We didn't care how we got them as long as we got them. They would walk us through really fast, and they would try to show us. We were doing screenshots and trying to record them in WebEx. 

Since a lot of the business process owners are onshore and the developers for the automation are offshore. It was very challenging trying to find a time when they could get together. This product allows them to do the capture on their own time. It can be very quick. They just send it over, then they're done. They don't have to think about it anymore, and it's documented well enough that we then don't have to spend a lot of time coming back to them. It just makes the whole process more efficient.

I would like more reporting in analytics. There is a lot of manual work for us as program managers and test managers which has to do with supporting our value statements. E.g., if there is some way that we could systematically capture how long it is taking for automation processes to execute, then we could insert some notes as far as here is how long it took for them to do the manual capture. Then, we could calculate time saved and have a formula for savings. 

If they have some templates that we could all partner, there are a lot of customers who have created their own, but with the new companies coming onboard. Do they create them on their own or do they try to leverage the best practice within our customer community? There is more we can do here.

There are no issues so far. We haven't had any problems with the tool not being available for us when we need it. 

We had trouble with an upgrade once, but there was an immediate response on their side. We had a very technical resource who helped us get past it quickly. So, there has never been anything which has really stopped us from working.

Because we started early with the tool, it took us a while. I feel like we almost glued things together as our needs became more. The capability that was delivered by Worksoft was there. Therefore, there hasn't been a time when they have been behind me when I have needed something that wasn't available.

I am expecting it will the same in the RPA space. We will grow together because of our close partnership, and if there is a gap, I can work with them to figure out what the best approach is to close it. I think we will be able to stay with the tool for a long time because of its scalability.

I am aware that they have some challenges with some of their support resources, especially offshore which is very common. I don't think this is specifically a Worksoft issue. It always seems to be a software issue, and I know that Worksoft is aware of this and they are trying to make some improvements. 

If ever I need to escalate something, I never feel like I'm stopped. I always feel like there is another level where I can go and get support. We have never had an issue which has gone unresolved for a long time. We try to follow the process, but since our team is so experienced with the tool, if we can't fix it and their support can't fix it either. then it gets escalated up through the chain, getting somebody whose pretty senior with the tools to help us.

Our initial goal was regression testing. It was really expensive. It was throwaway work. We always had to outsource it. It overlapped other test cycles within a project. So, all the functional business folks were busy. It was something that if we left it up to the project resources, they didn't do a very good job with it. We would bring in manual testers almost literally off the street. They didn't know our processes. They ended up having to interact with our project resources anyway. It was just a mess. It was inefficient, clunky, expensive, and the quality was poor. 

We knew that we had a lot of SAP implementations coming up because we had acquired several new companies. So, we made the decision at that time that we needed to automate regression testing. That was our first initial goal, and we've hit that. During our last major SAP implementation project. Our regression coverage was at 90 percent which is pretty much the top you can ever really expect. Now, we are looking at other use cases.

The initial setup was a little rough in the beginning. It was so new for us. The whole idea of automation was new that trying to get the tool setup, internalizing all the best practice training and everything that came so quickly, was a lot to try to digest. Thus, we ended up asking if we could spread the mentoring out across a few months. This seemed to work better for us.

We purchased the software, then we found a vendor on our own to help us with the development.

It has saved us significant time. I have an entire dashboard that I use to showcase to everybody the amount of manual hours that we have saved and how that equates to dollars saving.

On our last big SAP implementation project, we inserted an automation resource into the beginning of the project. Between automating regression processes, data staging, and using our automation to help repair cutover and conversion issues, we saved the project about $1,700,000.

Worksoft has paid for itself fives times over.

We evaluated Micro Focus UFT, Selenium, SAP CBTA, and Worksoft.

My main focus was an SAP automation solution, and Worksoft was really good in that space. They were an SAP partner, but I recognized that I wanted something to be more scalable across other applications, and that ruled out the SAP solution. We liked the price point of Selenium and some of the open source tools, but there are risks to something like that. You don't have as much control, and there are always security concerns. Our internal teams weren't excited about that, as they are not great with SAP. We already had a lot of pain points with UFT. It took way too long to develop processes with UFT. It required more coding.

I would recommend taking a slower, organic approach to automation. It is hard to insert ourselves into the projects. The functional resources, business resources, and process owners don't have a lot of time for us. They don't see the value initially. It is overhead for them and more work. So, you have to bite off small chunks. Show the value, then build up the trust. If you try to be too aggressive and force something down everybody's throats, they will barf.

If you have super strong executive support and it's a top-down, e.g., the CIO says, "You will do this or else." You may be successful. However, in that scenario, your failures will be noticed and made very public. If you take a slow organic approach, where you're just trying to be really helpful and free up time, doing little favors here and there, you build up confidence. Then, people support you more for your success.

Start with the low hanging fruit for the value. Build it up. Once you get a bit more expertise, then start tackling the more complex processes.

Worksoft is a great supplier to work with. They have never pushed back when we have had issues or questions. They have always been available to help us. They put us in touch with other customers that have done something similar to what we were looking to do. They set up user groups by region so we could get together. They facilitate a lot of good discussions. That's why I mentioned we continue to grow together: customers and suppliers. It's just been a great relationship. We don't get that with every vendor. So, when we have it we appreciate it.

It has been very easy to use, but I don't think every automation tool is for everyone. I don't think just anybody off the street can come in and use it. Maybe for some basic stuff, but if you really want to maximize the use of the tool, you need some folks who are really experts in it. 

We were able to really grow when we hit that inflection point: When we transitioned to a different vendor that we had doing our automation development. They were experts in the tool. That was when we started being able to deliver these creative solutions. That was when we were able to see the cost per automated script go down, because they were able to develop so much faster.

While it can be used by everyone on the surface level or to capture the business processes, to get more return on your investment, you have experienced resources using it.

My company brought on Worksoft to assist us with an SAP upgrade in order to make it so the testing was less stressful for business users through automated testing.

We configured Worksoft to setup our SAP testing environment after a refresh by regenerating all of our testing users. This sets them up with the right access, the right approvals, and everything on the back-end so we can do our scripts the next night.

The Capture 2.0 feature recording the process automatically as you go through. It saves everyone a lot of time. It allows the business to give IT the process they go through without having to spell it out so it can be recreated as an ongoing test.

There is a learn functionality where Worksoft learns applications that would be nice if Worksoft expanded its support for other applications that aren't web-oriented.

We have had some stability issues just getting stuff setup. It seems to be pretty stable outside of the web UI. We have had some issues with our testing running into nightly backups of database backups, etc. That has caused some issues, but when we get everything ironed out in a nice, controlled environment, it seems to be pretty good. The web can be a bit finicky sometimes, but it's just that the response times aren't always the same. So, it's a little harder for it to be resilient.

The scalability is really good. We had just talked with Worksoft about if there were any limits in terms of how long your scripts can run for or how many can be running at one time on the database. It seems the only concern is the hardware that you are running it on.

We ran into some issues with the version that we were on during the initial setup. We ran into a bug on one version, then they upgraded us to a new version, and we got hit with another bug. So, they had to put us in a beta. That was a little frustrating. However, besides the bugs that we ran into, the install was pretty straightforward.

It saves us time. It makes it so we are not as worried about changes which are going into the system. We know that we have nightly runs to ensure that things are working. In general, with upgrades, we can always rely on the testing to make sure that certain business processes are working. If they do stop working, we know when/where and can tie that back to changes in the environment easier.

Our eCommerce platform is Hybris. We run end-to-end tests where we place orders in Hybris, then we validate the order in ECC. Additionally, when an order is placed on Hybris, our QA environment has a lot of things which the SAP analysts have to prepare to get an order ready, so it doesn't clutter up the system, such as creating deliveries. Worksoft can do this for us as well.

Hybris is out most modern application. Our point of sale system is web-based, and it is in web form. We are on Azure. One of the things that we've been able to do is use Jenkins to put our Azure machines on business hours. We tell them to turn it off at 5 PM, then we tell them to turn it on at 7 AM. This has saved us about 62 percent of computer operations.

Prior to Worksoft, there were three different individuals within the company who worked collectively for about three years trying to automate just one of our smoke tests for our point of sale system. A lot of them got pretty far, but they weren't able to finish. However, with Worksoft, we have been able to automate six of our smoke tests in four months.

The most valuable feature is having a no-code solution for automation, so our QA team and some of our business users can work on automation. Then, they don't have to be developers.

Most of our SAP analysts use LiveTouch. They use LiveTouch along with prebuilt components. Our QA team uses LiveTouch when they need to add things.

I would like to learn how to get better logs for their support team.

It is fairly stable. We have run into some intermittent bugs off and on that we can't explain. Since they are typical Window's stuff, you just kill them.

When it is unstable, there will be times when a test that we are running in Certify will just stop, and it will say, "Aborted." There will be errors. There will be no explanation as to why it happens. It has now happened maybe one out of 20 times. When it happens, I just tell our QA team to stop Certify and restart it, hoping we don't see it again.

Scalability is throttled in part by the system that you are testing. So, how much testing can your system handle? 

I think they came out with a different type of licensing specifically for testing. Therefore, you don't have to use a more expensive user license, you can use an automation license. So potentially, if we had 100 use cases, we could spin up a 100 different machines, have them all run and be done in five minutes. That would be the goal, but I don't know if that would actually succeed or not.

For the couple of the issues that we were really scratching our heads over, we were in communication with the technical support several times, but they never got back to us. The issues are not critical because they're not really blocking anything. They're just annoying.

We needed a first step in order to get into DevOps. The first step was being able to automate our smoke tests and regression tests. They are tests that we use to make sure that our SAP environments are viable and our point of sale system. We chose Worksoft because they were the only people who we could find which were capable of automating SAP right out-of-the-box.

We needed a faster feedback loop. We have a third-party who develops our Hybris application for us and wanted to be able to hook into their Git repository, so when they push a new version, it would automatically deploy and run our smoke tests. Then, I can know within ten minutes if it works.

The new environment was pretty straightforward to set up. There were four servers, and maybe a fifth one, if you wanted to have a separate server for automation testing.

Some of the integration depends on the subject matter expertise on your team. How well do they know ECC and their processes? Then teaching them how to use Certify to build out their processes. So, on a scale of one to ten, it is probably a seven if you are not familiar with some of the development principles, like looping. If you are not familiar with them, then it will become more difficult to build out processes needed. This is just understanding the methodology of doing certain things, not Worksoft specifically.

The IT department and I worked with a Worksoft deployment engineer because of all of our Worksoft infrastructure. She walked us through setting up the database in SQL and the MongoDB with Worksoft Analyze. It was a pleasant experience. Most of the issues that we ran into were because I did not know something.

Because we haven't built out our suite of tests yet, we haven't saved that much time. However, we know that it will allow us to save a lot of time and money, because once we are fully DevOps, we'll be able to spin up and spin down our systems on demand. Then, we will know within 30 minutes whether the system deployed successfully or not.

We did not evaluate any other vendors. We didn't find anything else that did what we wanted.

if you can use Azure or AWS for your Worksoft infrastructure, then use that for ease of deployment. Once you have your environment, then you can save it using Infrastructure as Code. Thus, if you needed to rebuild or repurpose it, you would be able to do it.

We haven't taken advantage of all the current functionality.

We hardly use the Capture 2.0 feature at all.

We have been using Worksoft Certify for ten years.

We do automation of both SAP and other applications: One of them is our new grants management system. Originally, they wanted to do everything manually. Now, they highly rely on us to do testing in very short periods. 

We do web UI testing of modern applications. In our environment, we have SuccessFactors, plus SRM through Fiori. 

We also do performance testing and end-to-end testing of packaged applications, like SAP, InfoEd, and OnCore.

The most valuable features right now are the Capture capability and versioning. The Capture 2.0 has a lot of flexibility.

The easy of use and ease of integration of Worksoft Certify are very good.

We are looking for some enhancements on the Capture 2.0 tool. This would give us the ability to control it directly, like we could with Capture 1.0. Right now, Capture 2.0 doesn't really work for our Business Analysts. Because they don't work with it all the time, it's a little complicated for them to stay up to speed on it. With Capture 1.0, we wrote a wrapper to make it easier for them to use, but we can't use that wrapper with Capture 2.0. So, if Capture 2.0 gets enhanced, we'll start using it. For an enhancement, we want to be able to start and stop recording through an API. Then, we want to see how many steps have been recorded through the API.

We do a lot of test maintenance because they are constantly changing the applications. This is one of our biggest problems that are constantly making changes and switching products. For example, we used to use the Supply Relationship Manager. Now, all those tests that we build there will be replaced when we go to Ariba. All the old SAP GUI stuff for HCM, when we got SuccessFactors was thrown out, then had to be redone with SuccessFactors.

It gives you the ability to have a more structured environment for tests. It is not just recording of key strokes. It is more systematized, more like a programming language. That is the biggest advantage for us. Because of its consistency, once the developers know one skill set, they can use it to automate any application.

It scales well. We have eight offshore people and four developers in New York. We can add developers as needed. With the offshore team, we've have at least 20 people trained on Worksoft, but eight people actively on the team now.

We have about a 1000 active users with about 5000 users total. However, this includes all of the employees and their self-service.

The technical support is very good. They have always been a great help. With a lot of companies, you can't even really talk to the developers that someones provides you. With Worksoft, you can get a hold of the developers and work with them. It is all very timely.

The initial setup was very straightforward. We just build the database and installed the client application on the system. There was nothing out of the ordinary about it. It is not like setting up a lot of the IBM products or the Quality Manager, where there were so many different modules. There was just one module to set up.

During our yearly upgrades, we have now gotten them down to ten days or less. We have Worksoft run all our integration tests, where it used to take probably six weeks to do that manually.

As far as maintenance testing, it has saved us time. We find problems a lot of the time that they aren't aware of. This is because we run the tests even though they don't ask us.

We evaluated Micro Focus QuickTest Professional (QTP).

Properly staff the testing team before they attempt to do automation. Be aware that this will not be a one-time overnight process. We tried to automate everything in two months with eight people, and it was impossible.

We have been using it for so many years that we are really very happy with it.

We will be converting to Ariba for purchasing, so that will have to be automated.

Worksoft Certify is being used to run automated weekly regression tests across some of our primary SAP systems in line with our Change and Release management strategy. These tests run every weekend without fail. The results are reviewed on every Monday morning to check for failures and to analyse if any failures are associated with the changes scheduled to be transported to the production environment that week. Failures (if any) are fixed and the tests re-run before transporting the associated changes into the Production environment.

We also utilize it for projects that need extensive business-user testing and functional testing. There can be testing requirements which come at short notice which can take three to four weeks of manual testing effort. By using Certify, we have been able to bring timescales down to a few hours of automated testing effort.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems which will demand extensive regression testing. 

We have never had any systematic regression testing regime in the organisation. This has helped in building an automation framework across our SAP application landscape, thereby introducing mandatory regression testing across all our key systems and improving the overall quality across our production systems.

From an audit perspective, results generated from Certify (BPP reports) provides detailed test evidence which is also being utilized for internal training purposes/training guides, etc. The BPP reports also provide details on failures along with screenshots.

We have a variety of complex systems in our landscape, one of them being the Openlink Endur which is a commodity trading and risk management system. We are currently building an automated regression test suite to support application testing for Endur.

Our weekend regression tests are performed in 'lights-out; mode. Tests are scheduled to run at a certain time over the weekend using the Execution manager functionality. Usage of Certify has also prevented some major defects going into Production and we have seen significant savings in all manual testing activities as the business users/functional teams are getting more time to perform 'value- adding' activities.

Post our recent upgrade to Solution Manager 7.2, we are currently in the process of implementing the Test suite functionality and the integration of the same with Certify. We expect Solution manager to be the single source of truth bringing out all the results from Certify which is going to be extremely beneficial from an audit perspective. We have already implemented the integration of Certify with HP ALM in our landscape.

Moreover, we have this reusable asset now which can be run frequently to support all our projects and change requests across our legacy SAP systems. Even last-minute testing requests are being accommodated by utilising the automated regression suite without any dependency on business users/functional users for their efforts. We use it across the multiple projects which need immediate assistance and for our weekly regression cycles. To give an example of a recent project which was a major platform migration from a Data Centre in Asia to Europe which needed extensive Disaster recovery testing and Functional testing/User acceptance testing. The initial testing estimate was approximately five to six weeks, however with the use of Certify we could do extensive testing in less than three hours saving many weeks of manual testing effort.

It is very user-friendly with an appealing UI, unlike a lot of other automation tools that we have evaluated. With sufficient training and adoption of best practices, the tool will certainly help organisations to successfully implement an automated testing framework and eliminate manual testing activities.

The fact that it can be used across SAP and non-SAP applications (including web-based apps like Web Dynpro) is a big advantage for us because we have a variety of SAP and non-SAP applications across the Johnson Matthey IT landscape. Being a 200-year-old organisation, our variety of legacy systems have a lot to benefit from the use of automated testing.

Certify has many interesting features, e.g.: 'PRIMO' which is the image recognition functionality is a life saver in instances where Certify standard functionality cannot identify and learn objects within certain legacy applications.

Regarding end-to-end testing of packaged applications, Certify is primarily used across our SAP application landscape and the Openlink Endur (commodity trading and risk management system). We hope to realise more benefits by implementing Certify across our wider application landscape over the next few months.

We have been using the Capture feature, although not the latest version, the initial version, for process captures was used to create our test designs. It has been a life saver in many instances, without the need to spend any extra effort to create test designs and captures. The test steps get captured in the background which generates an XML file which can be easily imported into Certify, creating the basic test structure which can be improvised/modified to make it a repeatable reusable test. In terms of the amount of time it takes users to create documentation automation using this feature, it is the same amount one would spend to do a manual test. While a person is performing a manual test, Captures are automatically generated in the background. We have used it extensively to build our test designs.

We have requested for some minor new features which Worksoft is considering.

The PRIMO image recognition functionality has room for improvement, especially around its ability to work with java interfaces, Execution manager scheduling, etc. as we have observed. As we explore more of our legacy systems, I am certain there will be a need to use more of the PRIMO features to learn the objects.

Overall from a SAP perspective, it works almost seamlessly.

One to three years.

The version of Certify that we are using has been mostly stable and we have rarely encountered any problems. Our weekend regression test failures are often associated with environmental/system performance issues and not related to the stability of Certify. I have been happy with the overall performance of Certify and how it has helped to optimise our tests.

I am confident that Certify can scale to fit our automation testing needs as we expand the current automation testing framework across the wider Johnson Matthey application landscape. We are also exploring options to identify potential areas where Certify can help support mass data uploads, etc. to benefit other teams in their day to day operations.

We have several concurrent users accessing Certify in our environment, primarily automation engineers, test engineers and tech managers.

Our interactions with technical support has not been the best always and there is room for improvement especially with respect to the time taken to respond to cases. However, with the right contacts and reasonable escalations we have always managed to get quick attention on our issues.

We have always been heavily reliant on manual testing and as a result, regression testing was not systematic and we could never think of implementing frequent weekly regression test cycles which was challenging. We decided to go ahead with automated testing and use Certify because:

1. Manual regression testing takes a lot of time and resourcing is always a challenge.
2. Regression testing not being systematic, the quality was very difficult to measure as we did not have a standard set of manual regression test scripts/sufficient documentation.
3. There was a delay in our time-to-market because all the testing was being done manually and there was no way we could accommodate frequent, weekly, regression test cycles. That meant high business risk, that we would have more defects in the production environment/ more associated costs.

We had all these challenges and we started exploring options to mitigate these risks and automation was identified as the way forward, nearly two years ago. We evaluated various automation tools in the market. It was critical that we had to identify a strategic tool which would cater to our SAP and non-SAP application landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

We went through a lot of initial challenges, mostly around internal resourcing issues. Looking back, I am happy to say that we could overcome these challenges and have managed to successfully implement an automation framework using Certify.

Early in 2017, we decided to go ahead with Worksoft Certify post evaluation of multiple automation tools. Our initial engagements with Worksoft consisted of several onsite workshops to explore the tool in detail along with technical feasibility assessments across our application landscape. These engagements were extremely beneficial and it gave us the overall confidence to adopt Worksoft Certify as our strategic test automation tool.  

We did a pilot implementation with Worksoft to see if we could take this ahead on a large scale before embarking on the major project to build the automated tests. Some key processes across our critical SAP systems were identified as candidates for this exercise. Test designs were created with support from the functional teams and taken ahead for automation build with Senior Worksoft consultants and our internal resources. This 7-week Automation Roadmap Engagement exercise was extremely successful and we learned a lot of lessons from it which helped us plan the next big phase of the automation roll out. It gave us overall confidence across the functional and management teams which subsequently led to securing the appropriate budget, etc.

One of the biggest lessons learned from this engagement was around the ways to structure our teams. This led to us going ahead with a Managed Services model with Worksoft. We have an offshore based Worksoft Automation Services Factory team who helps build our automated tests. The team can scale up/down based on our automation forecasts.

The automation deployment is still ongoing. The initial phase was completed across a five-month span. Currently we are rolling out the second phase of the automation build focusing primarily on our global Unify solution and the Openlink Endur application.

Regarding implementation strategy, we followed an agile two-week sprint approach. Our functional teams continuously created test designs and these were fed to the Automation Factory every two weeks, who in turn developed the automated tests. This was the most practical model, which worked well in our environment.

At its maximum capacity, we have had approx. 10 to 12 automation engineers in the Factory team. Our functional teams are spread across multiple global locations and we had between 3 to 6 resources working on test designs liaising with the business users as required.

From a script maintenance perspective, we spend an average of 4 to 5 hours every week with the current asset of nearly 800+ tests.

We have always worked directly with Worksoft, along with support from our internal resources. Worksoft has been delivering excellent services through their managed services model.

We have a res-usable re-runnable asset built which is saving a lot of time across the functional teams/business user community.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems, which will demand extensive regression testing. 

It is expensive compared to some of the other automation tools in the market. However, the benefits and ROI has proved that it has been a good investment.

We have concurrent licenses.

SAP TAO and Micro Focus UFT.

It was critical to identify a strategic tool which would cater to the testing requirements across our SAP and non-SAP application (including web based apps like Web Dynpro) landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

It is a great product and we have not seen anything which cannot be automated till date in our application landscape.

It is important to do sufficient technical feasibility assessments before deciding to go ahead with Certify and equally important to determine the best implementation approach which will work for your organisation. Functional teams/business users' buy in is critical as the test designs cannot be created without their continued support. Adoption of best practices around naming conventions/folder structures etc. will help in easy overall maintenance of the test assets, which will also help with the generation of development and execution dashboards/overall reporting.

I would rate Certify at eight out of ten. Worksoft has always been very supportive and responsive to our needs and this has certainly helped us achieve our initial milestones successfully. I am extremely proud of what has been achieved so far and looking forward to expanding the automation framework across our wider IT application landscape over the months ahead.