Worksoft Certify Reviews

Its primary function is test automation. We record the test procedures and execute them in an automatic way. It gives us reports, in a centralized way, of all the information that we need to manage test automation for the ERP system and a few other systems as well.

We moved from manual regression testing, once a month, to continuous testing every second day, with the same scope or even an increased scope now. We don't need to focus on the manual regression tests, so we've freed up hands for support activities and new developments. I don't think that it has changed how our organization works directly, but we have become more efficient and more flexible.

In our estimation now, we save about 160 hours of manpower every month because we use Certify. It's 80 hours of robots executing the tests, and they're doing it roughly three times faster than humans. To do the same manually, and keep up with the general pace of development, we would spend 240 hours on testing every month.

In terms of codeless, end-to-end process automation across packaged applications, technically speaking there is some programming or development, like you would learn in middle school. But you don't need to be a professional in any programming language to run it. That's how we've done it. We don't deploy any additional software packages with it. We haven't developed anything else, other than using Certify steps to run end-to-end test automation.

Nevertheless you still have to have a mindset of a developer or an engineer. You need to have understand cycles and other primitive functional blocks. Still marketing-wise, I would call it codeless due to its simplicity. 

SAP is our core and the people involved with it are not always developers. Some of them are analytics or system experts. They don't know how to write C++ or Python or Java. But they can do testing automation in Certify themselves without help from a centre of competence (CoC). The only time CoC will participate is when truly complicated logic required such as runtime variables staking.

In usual situations, the skill set to start doing test automation needed is much lower.

There are a few aspects that are valuable. First of all, there is the screen-grabbing where you can pick up objects and it automatically recognizes them on the screen. Worksoft call it Certify Capture or LiveTouch. It's automatic capturing properties of the UI objects on the screen or on a mobile device for use in test automation scripts.

Other important feature is the modularization of the tests. That's an important feature not only of Certify but of IT tools overall where you create and reuse the components. Our test scripts are done with a "Russian doll" approach. That's standard for modern IT, however sometimes you see IT products without such functionality. Worksoft Certify's ability to build tests and reuse them is done pretty well and balances simplicity with flexibility.

Due to it, the learning curve improved even further allowing non professional test developers to plug-and-play test scripts. Such approach allows you to have that nice, modular end-to-end test. Can it get better? Probably not without sacrificing some other nice features and simplicity.

In terms of the core functionalities important to us there are the actual coding, great test data management, execution and reporting. Those are the basis, which are shared across many test automation tools. 

We also value that the test execution and reporting are stored in a database. We can extract the results and get the data out more easily comparing to text log files. That's different compared to other tools on the market. In Certify, we have a structured test execution and overall structured data.

Last not least is multi interface support is important for our applications landscape. With certify we can test SAP, Net, Java, Web and Android applications with one tool.

Certify's web UI testing abilities for testing of modern applications like SAP Fiori was good when we started and they developed it to be even better. We all know that web applications can change objects in DOM quite fast and it breaks tests. To counter it Certify has made object recognition more flexible and generic, so we don't have any troubles.

For standard and UI stable applications like SAP ECC 6.0 we have no trouble at all due to predefined objects and rich library of methods. That means we don't need to spend time writing a library, which could happen in some of the open source frameworks.

One caveat is that if you start running models in different parts of the end-to-end process — when you really try to hit the sky and make everything automatic, to cover multiple supply chain tiers processes in one e2e test, or similar processes that are really complicated — then tool simplicity turns into disadvantage. On other hand it stops us from unnecessary over engineering the test automation framework.

Architecturally, because Worksoft specifically built a database-oriented application, you essentially store code in the database. Git and text files orientation is more traditional approach with boundless set of tools to control versions, manipulation and analysis. But at the same time, Worksoft supplies us with their own version control inside Certify that has sufficient functionality for now. 

When it comes to mobile testing, we have a small bottleneck there. You have to buy third-party separate licenses if you want to test on a mobile. Business wise we see room for improvement there, although it's that really critical for us. 

None of these are showstoppers for our operations. Worksoft proved to delivery significant improvements in last 3 years and more we wait from 11.5 version. Overall, we are quite well covered with test automation related tools and nothing special is needed.

We've been using Worksoft Certify since 2017.

It's stable. Our approach is that we don't deploy the latest fresh version to production straight away. We wait for a few patches and follow with upgrade instructions. It doesn't matter if it's Certify, DB server, Linux or anything else.

Only case I could recall related to stability problem is a one mess-up in the database during over-few-versions upgrade about three years ago. It required some database cleanup with scripts provided by vendor support.

But looking at the overall stability, it doesn't go down or crash. SLA is 100 percent for us.

Because it is three times faster than manual testing, we can test more with fewer people. That gives us operational scalability already. Platform also has the Execution Manager tool if we would like to have technical scalability via a cluster of virtual machines. In addition Certify itself supports multi-agents, APIs, and integration with Jenkins and other applications including PowerShell. That means we can also develop scalable setup ourselves

As for overall scalability of Certify, licensing is the main bottleneck. You cannot scale more than you have licenses for. You cannot uplift your digital workforce just for one month and run 300 nodes strong cluster for a big upgrade tests. It is not different for majority of IT products, we don't see much of a problem for now.

Certify is our main tool for test automation for ERP, for PLM, and HR. It's covering two major applications that are at the core of any big machinery or manufacturing corporation. We continuously increase the scope, because systems are developed and extended.

We use their enterprise-level support and it pays out. We mostly use it for tricky situations. We often talk with 3^rd level, with architects and other experts behind the certify. It gives us good insights. Separate gratitude goes to active key account management on vendor side.

As for our improvement requests, we speak with the community and we speak with architects and provide necessary information. We don’t know do they take it into their backlog or is it just coincidence that Worksoft have provided most of things we asked for. In any case we get what we require for our testing, we in constant contant and we feel that we are listened to.

We did not have a previous technical solution.

The initial setup was pretty straight forward, in general. There is a database, there are clients to install, and you need a licenses server.

The key thing that you need to know is what you are doing, how you do test automation, in general. That's above the specifics of any tool. You must know how to slice system-under-test into functional components, make the right model for reuse. It is important to break down end-to-end testing into smaller reusable items. That's not trivial, and you faced such challenge with any tool. You have to know what you're doing.

The deployment included one of week training and then a proof of concept for a couple of months. We ran typical end-to-end process in the proof of concept and found our ways to deal with above mentioned slicing, development guidelines and how to establish roles. There are useful guidelines provided by vendor as well. Overall it took about one quarter from the GO decision to start running regression tests.

Our approach to testing is as a "safety net," so we don't need to have end-users who run it and hunt for new defects. It's just continuously testing given scope and raises a red flag if something goes wrong. Such approach secures an immediate feedback for the development team or for the quality assurance team.

End users are not involved into testing usually although we could run hybrid testing with a mix of automatic and exploratory tests.

We didn't work with any third-party. There was initial training from the Worksoft itself for one week. That was it.

The return on investment, when we count the 160 man-hours we save monthly, will take about seven years for our scale

License model is a traditional one. As with any other manufacturing company, the software testing is not our core business, it's not in our interest to invest capital into the licenses. It would be preferable to have a pay-per-use model. At the same time, it's a fair game, for now. 

Aside from their standard licensing fee, if you want to have enterprise support, there's an additional cost.

We have done a few comparisons and we're continuously looking at the market to see if there is anything better, cheaper and more suitable. Market is changing a lot, RPA solutions expanding to test automation market, Robot Framework ecosystem is thriving and some other established vendors are operating. We're keeping an eye on it for evaluation. So far, cost-wise, feature-wise, and in terms of the learning curve or skills requirements, we are good with Certify. From our comparisons now, we would pick Certify again.

As always, you have to know what you're trying to achieve. You also have to think about how do you model your system-under-test; how it is written, the quality requirements and standards. The key benefit comes from reusable test scripts. 

When it comes to Certify itself as a tool, keep your eye on objects. If you map something, keep a good inventory so you understand it and you don't put all the objects in one big basket.

For me, the biggest lesson from using Certify is that you can do quite interesting and complicated things with the codeless approach. You don't really need a complicated programming language to do testing. 

Overall, I would rate Worksoft Certify an 8 out of 10. Of course we would like it to be cheaper and more powerful and developed faster. But are there products that I would rate a 9 or 10? I don’t thinks so. It's at the top of the market from what we know about the market.

We have developed some end-to-end regression testing scenarios that we've found pretty valuable, so we have created a bunch of processes in Certify, linked them together, and we use them every week - sometimes more than once a week - in regression testing.

We used another tool for many years. It became unworkable because of the length of our scripts and how many of them there were, and how they were linked together. They became cumbersome in the other tool. It's much easier in Certify, and Certify can handle them, no problem.

In addition, our organization is implementing Agile, we're moving towards continuous development, and I don't see how we could do that, in any imaginable way, without Certify. We're able to import our changes weekly, based on the results in Certify. And we're confident that because of having tested the main business processes, fairly rapidly, within one day, we can tell whether the imports are going to break anything.

It has absolutely enabled us to automate and save time. The weekly imports of the changes allow the developers to plan on a weekly cycle, which increases the speed of their development. They don't have to wait for a release or anything else, they can test their changes quickly and get the results the next day. They know that they're able to import with no problem.

Finally, it has helped us cut test maintenance time. 

The idea that it's not language-specific is really nice. Keywords and the drag-and-drop functionality are great. The ease of use is superior to anything on the market. It's very easy to integrate. We've been very impressed with the tool. Because we primarily use the configuration with SAP, the integration is pretty seamless. But we have used our own in-house VB app as well, and it's worked very well with that. We've really not seen any problems whatsoever with integrating.

We love the Capture 2.0 feature. It seems to work very well. As for how long it takes to create documentation using it, we do not get into the documentation so much. That end is not as useful to us. But it's built-in if we ever needed it. We're not USDA or anything like that so we don't have a super need for documentation right now.

Also, the ability to work with the data, with recordsets, and plug those into the scripts is very easy and very powerful. We use it extensively.

In the past, when we've tried to automate some of our web apps, it has not been as robust. If there were one thing that could be improved, it's interaction with web applications. The issue we were running into is that it was harder to identify the objects than it is with some of the other architectured applications. That's the nature of the beast with the web as well.

It's very stable.

We have had some issues. We would be unable to log in, in certain situations. But they've all been self-inflicted, changes that we've made on our side that have prevented us from being able to use the tool at times. Once we got those resolved, we were fine.

It's fairly easy to scale, which is a nice thing. Once you create what I will call a sub-process, if you want to use that sub-process in many other processes, it's really easy to use. For us, that's what makes it scalable. You can use that same process wherever you need it. The use of the recordsets just allows us to be able to change the data that make it unique and that make it easily maintained. It's very easy to scale. It simplifies our workflow.

Worksoft technical support is very helpful, very knowledgeable. Whenever we've had an issue, they've responded very quickly. We don't actually have very many tickets, but whenever we've had them in the past, I've just gone into the portal and I get an email back, usually the very next day. I've never had to escalate an issue.

We were using Micro Focus UFT. It became completely unworkable for us. Our end-to-end processes were just too cumbersome for the tool to handle. It got worse and worse to the point where we had to say, "You know what? We have to change tools, this is not helping us." That's when we investigated Worksoft, and we were very pleased with how it worked.

The initial setup was pretty straightforward.

The last time we did it, for the upgrade into version 10, it just required me and one other person on the database side, and then the technical person from Worksoft. It was fairly easy. It took just a couple of hours. At that time, we were just upgrading. The basic architecture was already there so it didn't really require a project plan or anything like that. Once we got it set up, it was just a matter of migrating what we already had in UFT.

There have been several times where it has highlighted an important issue. Some of the defects we've found have been high-impact defects that would've really been costly had they made it to production. There are other times where, because we were able to test with Certify, we knew within a day whether there were gaps in the way we configured a change, things that we had missed that we wouldn't have been able to find if we didn't have the ability to test quickly.

That one defect we found easily saved us $1,000,000. That was just one. Over the years, the amount of money that it has saved us is certainly in that range.

The ability to test quickly has enabled us to develop quickly. We've been able to capture lots of savings in terms of projects that have been delivered faster because we can test faster.

There are savings on a lot of fronts because of this solution.

We would purchase more licenses right now if they were cheaper. Pricing is a little bit of a hindrance.

We didn't look into any alternatives.

It's a highly powerful tool. It's very customizable. It's not a cure-all for everything, but if you want to do end-to-end testing, regression testing, it's a great investment.

We use Certify for end-to-end testing of packaged applications. We have implemented almost anything that touches SAP, using Certify. When C4C came out, the customer application, we regression tested our existing suite to make sure that nothing would break. We anticipate doing the same thing with Success Factor. At the moment, we don't use Certify for web-UI testing, but we're planning on implementing some of that, coming up.

Since it has been up and running, we've had three people maintain it: Myself, I'm the principal QA person, and we have two offshore partners whom I've trained on Certify and they are now helping us execute and maintain the tests. It requires full-time maintenance. We have plans to expand the reach of our automated testing, so we plan on adding more people. We are the only three using Certify in our organization at the moment.

It tests our core business processes but we still have many core business processes that we would like to add to that, to validate if they work, before we send changes through every week. And we would also like to increase the speed at which we can add changes; not just once a week, but eventually daily. We plan on increasing our resources from a manpower standpoint and also from a technological standpoint. We're just going to try to do that as fast as we can. There are a lot of business processes that we would like to add, a lot of apps that we would like to add. The business side has continual, increased demand in terms of things that they are working on and they would like to automate and not test manually, so there's a lot of demand on us right now.

I would rate Certify at nine out of ten. I rely on it every day. It's a great tool, and any problems that we have are hardly ever attributable to the tool itself. It's always some other factor; the way we're using it, or some external factor, which is the problem. It's nice not to have to worry about the tool being the issue. We're very enthusiastic users.

We have an SAP application, and for the automation of this SAP application, we are using Worksoft Certify. This project is in the healthcare domain. I get the requirements from the client. They tell me about the new changes that are going to happen in the system, and based on that, we do the automation. The client tells me about the scripts and new features in the application, and I create an estimation plan, which depends on the size and the complexity of the scripts and features, and share it with the client. We do the development activities accordingly.

We have two teams. Being an automation team, we are not very much on the functional side. So, we need to take some support from the functionality team. for test case writing. They are usually able to give us steps in Excel or something like that, and we are able to start with that. Otherwise, we have some sessions with the client or the functional team to discuss any issues or doubts we have. We also need data for developing the script. We get the required information about the data, such as if data is consumable and how to find new data, from the client or the functional team. Once the automation is done, we inform the client, and we are ready for the sign-up for the first set of the script. They do a functional review, and after they okay it, we do the final sign-off. We then move our scripts from the local sandbox folder to a production folder created in Worksoft Certify. We then use the script for regression.

In terms of its deployment, they provide the updates, and we install them on our VPS.

We are mainly using it for the SAP application, and for the SAP application, if you don't have any experience with automation tools, after a few training sessions, you can easily automate the scripts. That's because no specific programming language is used. All resources that I have are specifically SAP resources. They are not from the automation background, but after gaining the knowledge, they are able to develop a script, or when there is any issue while doing regression testing, they are at least able to understand the issue, such as whether the issue is in the code or data. 

We can use it for the web application, but we are facing stability-related challenges. The properties are getting changed. For example, when I am performing any operation on the text box but the development team has done some changes, our Worksoft scripts are getting changed. This is the main challenge that we are facing while developing tests for the web application in Worksoft Certify, where any changes in the backend are indirectly impacting our scripts. For the web application, there is a scarcity of resources. Unlike an SAP application that doesn't require much experience, for the web application, you require experienced people.

We also use XF definitions for the web application, but we can't create them on our own. So, we need to take support from the Worksoft team, and we have a dependency on them.

I have been working with this tool for the last five years.

It is completely stable for our SAP application. There is no issue. Once you have developed a script, you don't need to do any changes.

We are using it on quite a large scale. If we have a requirement for the new development, we have to use Certify. My project involves maintenance and new development. In the maintenance phase, we have regression testing. We have created a batch file, and when we are doing the regression testing, we are executing this batch file. So, we are using it for day-to-day activities. We are also using another tool from Worksoft called Worksoft Execution Manager.

The number of users working on this solution varies based on the project timeline and the number of scripts. For example, if the client's requirements need to be delivered in a very short period of time, then instead of four resources, I will take eight resources.

For any issues, we have to go to the Certify support team. They have a customer portal, and you just need to raise a request on the portal. Based on the priority, you get a response from them. Initially, they provide the solution through the ticket, but if the solution doesn't work, they also schedule a call. If the issue is not resolved in one call, they schedule more calls till they have resolved our issue.

I have used UFT in my previous company. Companies that have an SAP application or SAP Fiori prefer Worksoft Certify. For my current client, initially, we were using UFT, but because any non-automation resource can do automation in Worksoft Certify, we brought Worksoft Certify. Now, instead of UFT, we are developing everything in the Worksoft Certify. 

I am also familiar with Selenium. For SAP applications, Worksoft Certify is a very good tool. With UFT and Selenium, you need to create a framework, which is not required with Worksoft Certify. You can just go and develop an automation script. 

So far, I have used Worksoft Certify for the SAP application, the mainframe application, and the web application. For the SAP application, it is working well, but for the web application, instead of Worksoft Certify, it is better to go for Selenium. With the web application, we have the issue of values getting changed when developers make any change in the backend, and we also have a dependency on Worksoft for XF definitions. So, Selenium would be a better solution for web applications.

I don't have much idea about the pricing, but it is a licensed tool, and it is somewhat costly.

If you are considering this tool for your project or company, the Worksoft sales team would connect with you and explain what are the things that you need to perform. Initially, they give a demo, and after that, they discuss the licensing, etc.

It can be easily used by our non-automation resources with some training, but your team should ideally also have experienced people to be able to deliver within your deadline. For some of the development, you can also hire Worksoft resources on a contract basis. They would come to your location and help you with the development of a script in a short duration.

I would rate Worksoft Certify a six out of ten.

We bought the product in 2014, but did not have the right structure in place to use it properly. It wasn't until 2017 that we started to receive good value out of the product. Automation is now part of our strategy.

We use this solution for end-to-end testing of our packaged and SaaS applications. It is very much a part of our key strategy. We have it running end-to-end updates all the time.

We do web UI testing of SAP and ServiceNow. This was difficult in the beginning, as we had a homegrown product, and we had to do the proper object naming. With SAP, Worksoft does the object naming for us. This improved our website from 2014 until now. This is part of their process improvement.

Our business users are doing regression testing as their day job. This is an add-on to their daily work. With everything so pressured in the industry, automation takes the pressure of these users. This is essential for us.

It is not IT dependent. You can go to a business user, and say, "Just record what you're doing." They don't have to be very technical and can just do their job, then you can do the technical side or evaluation. This product helps us so much in capturing real business processes. This makes the overall process much easier.

A technical person thinks differently to a business person, and having to actually see what they do is the part that's so good for us. We can see the processes exactly as they are, not how we think they are. This makes a difference.

I would love to see more data mining modules. 

I want to see more stability in the Execution Manager.

There have been major improvements in stability since we started until now. So, it is becoming more stable. 

Also, we are being taught how to make it more stable, since stability is not just about the product. It is how you use the product. There are base principles that you have to keep on your data in the system. It is how you write the script and if you write the script to be sustainable. This will make it stable.

I love its scalability. I like records set that you can just add onto it. We are going into a new country every year and the ability to increase from test scripts makes the product so much simpler.

Their technical support works with us, which is good. Working with them has been a collaborative process.

Worksoft Certify was originally purchased for a product. We thought that it was not be used to its full extent, so we ran it through a PoC. Then, we decided that the product could work for us.

Ownership wasn't taken for the product during the initial setup, so it was a difficult process at first. Once ownership was taken, it wasn't that difficult.

We have seen ROI.

We have not looked at other vendors.

If you're looking at automation, it has to be part of your strategy. It's not something you can push bottom up. It's not something you can just do for a project. Automation is how we work. It is the thing of the future.

I haven't personally used the Capture 2.0 feature, but we've reviewed it. Worksoft has definitely made it simpler again by putting in comments and adding in some extra things. This has really helped, and our business users have been using it.

We have been using Worksoft Certify for ten years.

We do automation of both SAP and other applications: One of them is our new grants management system. Originally, they wanted to do everything manually. Now, they highly rely on us to do testing in very short periods. 

We do web UI testing of modern applications. In our environment, we have SuccessFactors, plus SRM through Fiori. 

We also do performance testing and end-to-end testing of packaged applications, like SAP, InfoEd, and OnCore.

The most valuable features right now are the Capture capability and versioning. The Capture 2.0 has a lot of flexibility.

The easy of use and ease of integration of Worksoft Certify are very good.

We are looking for some enhancements on the Capture 2.0 tool. This would give us the ability to control it directly, like we could with Capture 1.0. Right now, Capture 2.0 doesn't really work for our Business Analysts. Because they don't work with it all the time, it's a little complicated for them to stay up to speed on it. With Capture 1.0, we wrote a wrapper to make it easier for them to use, but we can't use that wrapper with Capture 2.0. So, if Capture 2.0 gets enhanced, we'll start using it. For an enhancement, we want to be able to start and stop recording through an API. Then, we want to see how many steps have been recorded through the API.

We do a lot of test maintenance because they are constantly changing the applications. This is one of our biggest problems that are constantly making changes and switching products. For example, we used to use the Supply Relationship Manager. Now, all those tests that we build there will be replaced when we go to Ariba. All the old SAP GUI stuff for HCM, when we got SuccessFactors was thrown out, then had to be redone with SuccessFactors.

It gives you the ability to have a more structured environment for tests. It is not just recording of key strokes. It is more systematized, more like a programming language. That is the biggest advantage for us. Because of its consistency, once the developers know one skill set, they can use it to automate any application.

It scales well. We have eight offshore people and four developers in New York. We can add developers as needed. With the offshore team, we've have at least 20 people trained on Worksoft, but eight people actively on the team now.

We have about a 1000 active users with about 5000 users total. However, this includes all of the employees and their self-service.

The technical support is very good. They have always been a great help. With a lot of companies, you can't even really talk to the developers that someones provides you. With Worksoft, you can get a hold of the developers and work with them. It is all very timely.

The initial setup was very straightforward. We just build the database and installed the client application on the system. There was nothing out of the ordinary about it. It is not like setting up a lot of the IBM products or the Quality Manager, where there were so many different modules. There was just one module to set up.

During our yearly upgrades, we have now gotten them down to ten days or less. We have Worksoft run all our integration tests, where it used to take probably six weeks to do that manually.

As far as maintenance testing, it has saved us time. We find problems a lot of the time that they aren't aware of. This is because we run the tests even though they don't ask us.

We evaluated Micro Focus QuickTest Professional (QTP).

Properly staff the testing team before they attempt to do automation. Be aware that this will not be a one-time overnight process. We tried to automate everything in two months with eight people, and it was impossible.

We have been using it for so many years that we are really very happy with it.

We will be converting to Ariba for purchasing, so that will have to be automated.

SAP HANA S/4 HCM, FI, BI, Payroll, SuccessFactors, Fiori, other web HTML interface applications.

Our company automates a lot of functional tests for SAP. The automated tests are included in larger integration tests of SAP transaction, workflows and end-to-end processes. Successful implementation and execution of Worksoft Certify tests has been faster than execution of the tests manually. With other tools, it was hard to get successful technical execution right quickly. Automation is not easy. Worksoft Certify makes it easier yet flexible enough to handle some of the most complex automation tasks. Every time steps are automated, test execution times are reduced and application problems are identified faster. Every time that the automated script runs (many times if developed early enough and applied with each change of the application being tested - in the current environment and leveraged in another environment), the investment of work made to develop the script is realized. The more times that the test is executed, the risk of problems going undiscovered is reduced. Testing time is shortened. Avoiding the tedious tasks of manual testing easily repeatable steps saves money.

Worksoft Certify provides the ability to develop test scripts with a low bar of entry among automation tools. It’s very simple to develop. However, great care must be taken to use the tool as intended, otherwise efficiencies and effort savings will not be realized. Like all automation projects, well-designed automation, with thoughtful planning and application, is necessary or you will unable to get back the return after spending the cost in investing time and effort with the tool.

Worksoft Certify provided outstanding feature support for SAP test automation which is unrivaled by other tools in the market. If implementing and need to automate SAP, first pilot the tool with a representative scenario, and then evaluate that retesting the scenario multiple times will ultimately provide compelling overall value for SAP test automation effort.

Worksoft Certify can better identify web test objects by providing libraries through its Extensibility framework. It has improved with it's latest version through Extensibility Tools support and additional test object attributes. Object identification is a necessary barrier for the technical execution of a test script, aside from the script's needs to verify business rules. Automation engineers must affirmatively answer the question: Is the application automatable?

If Worksoft Certify or any other test automation tool can get object identification down consistently and flexibly so that maintenance of web object identification (and hence the web test script) is easy and done with no execution problems, it will allow more scripts to be developed and applied faster with the human test automation staff resources on hand.

A good part of the technical test automation work is to make the script run and affect script changes when the application under test changes. When done right, the business part of automating the test application functions remains the same - leveraging your test investment.

Certify may not the best solution for your needs, but among the commercial and open source offerings in the automation tools space, it should rank very high. It is also a leader in rapid SAP test automation, and can be used as general all-purpose test automation tool for varied clients like PC, SAP, and especially Web.

Only one or two tools I have experienced could be suitable in place of Worksoft Certify. However those tools also carry their own set of disadvantages.

In terms of stability, Worksoft Certify is a polished and mature commercial offering. We rarely encounter problems with the tool. When we do, Worksoft is very responsive to the issue.

We also find the new features in version 11 of the tool to be innovative. The more friendly and fast UI, greater support for test object Extensibility support, Capture 2.0 functionality and global search are all market-driven features that benefit the bottom-line in creating and maintaining Worksoft Certify automation script assets. The improvements are welcomed and we look forward to even more such improvements to make Certify a top-tier competitor in the automation market.

In terms of scalability, we only have a few licenses. We manage and maintain the underlying database actively to avoid scalability issues. We are confident that Worksoft Certify can scale to fit our automation testing needs as we:

• Scale-up with many more automated tests of varying length
• Scale-out with unattended lights-out execution.

I would give technical support a rating of 8/10, 10 being the highest. Worksoft is very quick to respond and address customer problems identified with Certify.

My organization has been using Certify since 2007. We are committed and very happy with Worksoft. I picked up experience with Worksoft Certify when I joined my organization and leverage my existing automation skills to quickly become effective with the tool. I still use Selenium/Java/Python and was expert-level proficient several market leading enterprise QA automation tools.

The setup was done prior to my joining the organization. The existing work helps make my drop-in experience very smooth. It would be harder from the ground up, since there is a database and application server that needs to be installed and administered, and procedures and policies to define so that development is standardized (establishing naming conventions, specific ways to do things when there are multiple options available, etc.)

We implemented and developed in-house expertise. A vendor team would have accelerated and made adoption smoother.

We get at least a 2x return on investment (time, cost, labor) over manual efforts. When tests repeat execution the ROI can go higher. Tests often repeat when executed annually with planned annual system upgrades and additional time, labor and cost to maintain factored in. Including and reusing the tests in other testing activities such as sanity checks and regression cycles increases the overall ROI and investment value.

Conduct a proof of concept (POC) so that you understand what needs to be invested with Worksoft Certify. You will need to understand what sort of time you need to initially set aside to learn and apply the tool, and how that time can be leveraged to shorten continuing development of Worksoft Certify test scripts (processes) as facility in the tool increases.

• Someone needs to know to administer it.
• Someone needs to know how to develop the tests.
• Someone needs to develop a representative automated test script for a candidate application in your organization.
• That person also needs to execute the scripts and then report results of the scripts.

That script needs to execute several times. Now a comparison needs to be done with the time it takes to execute the multiple test runs against the same time it would take to do the same thing manually. In this way, you can now quantify your savings.

Now apply the need over the instances in your organization and compare that to the pricing and licensing to see how many instances of this effort would give you a return on your investment.

You will find that you will have many instances that would justify the price and licensing costs.

We evaluated QTP and Selenium. Worksoft Certify was chosen because of:

• The comprehensive coverage
• Extensive support with SAP
• Additional web test automation support
• Ability to develop tests fast and execute them reliably
• Technical support of the tool

The combination of all narrowed down our choice to Worksoft Certify. It was a great choice in 2007, and if evaluated today instead of 2007, it still would have to be Worksoft Certify since the importance of SAP support is priority and is significant over any of the other criteria and support provided by the other competitors.

Perform a pilot. Apply it to your app under a test with representative workflow. Understand the process of developing Worksoft Certify tests by doing it. Execute the develop tests multiple times. Can you live with developing the tests for Worksoft Certify? Is there value in the execution over multiple times quickly? Can you justify this effort over the cost of your current practices? If you can, then this incremental step can be leveraged and then take you further into more automation successes and favorable outcomes with Worksoft Certify.

Our organization usually tests on beta and production releases of Worksoft Certify so we are privileged to be able to work with the tool ahead of the general market. In that, we experience more issues that would not get out to production releases. Also even after release we work closely with Worksoft Certify support to identify and resolve Worksoft Certify functionality and may receive such priority support than a normal Worksoft Certify client would experience since we have such intimate technical knowledge, experience and relationship with Worksoft and the Worksoft Certify automation tool.

Because we are an active participant in a robust enterprise commercial process to help improve the stability of Worksoft Certify (staged releases to select customers to improve quality prior to some production release), I would say we’re contributing towards a good job of keeping up the stability of the automation tool by testing it first hand in our complex situation and environments and responding back with realistic and practical feedback encountered. We accomplish our organization's mission to develop and run diverse, varied and large amounts of Worksoft Certify processes automation and also assist Worksoft mission to develop and improve the Worksoft Certify automation software.

When speaking on stability, I refer to the stability of the Worksoft Certify software tool. Execution stability of the developed test scripts (Worksoft Certify processes) is a whole separate concern. Even so, Worksoft Certify also shines here in comparison with other tools. To really address this properly and clearly, you need to automate a small test scenario in an application that is to be tested and compare the tool script execution with another tool to see that Worksoft Certify achieves a high-level of repeatability and resiliency in script execution.

Even so, we do work with an occasional problem that comes out in production releases and experienced that the interaction and speed to which the problems are addressed and resolved by Worksoft Technical Support to be extremely satisfactory. I am not sure if this is representative of all of Worksoft’s clients, but we are very happy with their speed and service with which they handle our issues. This contrasts with the unsatisfactory paid subscription support experienced with commercial HP QTP/UFT and IBM Rational tools, and the lack of any support (other than searching the internet and relying on the user and development community) for open-source tools like Selenium and JMeter.

Without the interactive and engaging support, some of these problems would be extremely difficult to solve on our own. However Worksoft solutions were technical in nature beyond what a normal client user could identify and resolve on their own. Using the available and comprehensive technical support and educational services increases the likelihood of success and a positive outcome and benefit with Worksoft Certify. Successes with Worksoft Certify will come from meaningful collaboration to address automation issues rather than handling problems through independent or isolated effort (e.g. the problem could have been addressed with another customer and the resolution could have been worked out already. Worksoft support would be in the best position to know the possibility that a solution exists). 

This is typical of automation tools and the experience of other tools in the automation tool trade. The knowledge and experience is specific to the tools. As such, going to the source who has the most experience is the recommended strategy.

Our primary use case is test automation. We have an SRP solution called One1ERP on an ERP platform. We started the automation in Worksoft Certify, getting more than a thousand test cases automated.

Nowadays, we also have automation for a web application in our HR area, so all our HR processes on the platform, Pega. This started to be automated since November last year. We have 40 test cases automated on this application in one year. 

Improvement means for us that we have to be better in quality. Due to automation, you can run every automated test case twice a week. If you do it manually, you do it once per release. This is a quality improvement.

We always have to be more efficient. E.g., if we can directly automate test cases, so if we are using older test cases, then by just switch the company code (national company), we don't have to do it manually. We are able to directly automate the test. This saves money, which is always important for our top management.

We are creating a team an automation team, which will have up to 10 people/colleagues. This will be set up in Bratislava, and those 10 colleagues will work 100 percent on automation. Usually our SMEs do the test execution and try as hard as possible to have everything directly automated, e.g., 40 percent test execution and 60 percent automation. In the end, we want to have a pure automation team who is just doing automation, and have the testers on another team. After the tests are finalized, then they can give them directly to the automation team, or work on them together. That is a mixture. For those colleagues, it's really 100 percent automation

We are interesting to do better, year-by-year. At the moment, we are doing automated regression tests. The next step would be DevOps or artificial intelligence. Our programs should also develop in this way. We want to have automation everywhere where it is possible. Therefore, we need more options for these next steps.

We have used Capture, and it works with Worksoft Analyze. We had some experience last year in August because we started our schedule 1.0 and used Analyze in our tests. We were using the central site for one year with our ERP testing and were quite successful, but this was with Capture 1.0. Then, there was a change to Capture 2.0. In the end, there have been some challenges with the newer version. Therefore, the company testers, the local ones, do not want to use Capture 2.0.

40 percent of the test cases were finalized with automated capture and automated documentation, then the others were done manually. Because we have to create test nodes, we were asked to create a tool that automates documentation, which was Worksoft Analyze. However, with the switch to Capture 2.0, we had some challenges in the beginning. What we did afterwards, together with Worksoft, was we sorted through all the known bugs. So, at the moment, we don't have any known bugs open. We will retry this year in our central test first to find out if it Work Analyze is fine, then if it is okay, we will continue with the local test teams, as well.

On the Capture 2.0 topic, we were not satisfied, because we had a version that wasn't really tested from my perspective. Of course, Worksoft said it was tested, but we found a lot of bugs. In the end, our national company and local test teams did not use Worksoft Analyze because it stopped working. We have ten steps, then on the ninth step, it stopped working and we would have to do it again. So, they stopped using Worksoft Analyze and Capture 2.0. However, this is solved. Worksoft directly helped us to find out what the bugs are, and solved them. Then, we retested it. At the moment, we don't have any open bugs.

On average, it takes one day for Capture 2.0 users to create document. We expect it is faster, but you have to do it several times sometimes. You have to check the documentation that everything is fine. 

There have not been many issues at all, maybe one or two during the last three years. However, it has been quite stable.

Worksoft has good scalability. This is also the reason why we easily can automate for our new laptops.

Usually, we have one contact person for support, who then will contact the Worksoft support. Our contact organizes the people around her. While we have some direct contacts, and most of the time, we receive answers from them. I don't know if there is somebody behind them from the second or third level. 

We are forced to have efficiencies every year. We always get less budget and having to do more. So, we had to have some ideas, and our idea in 2017 was to increase automation. We had automation in place beforehand with eCut. However, in the end, we cut rates 10 to 15 percent. With Worksoft Certify, we had this campaign year with company codes up to 80 percent of the automation rate. This is much faster, and we are finding the defects earlier. In the end, you can save money and have better quality.

In three months, we created 1000 scripts with Worksoft. When the three years before with eCut, we did 450 scripts. This is where we saw a difference.

The initial setup was straightforward. 

In the beginning, it took a long time to integrate Worksoft in our landscape because we have a lot of security levels to fulfill. Therefore, it was not permitted to be install on the security server. It had to be installed through our data center. We learned step-by-step.

Due to security issues we could not use VPN tunnels, we had to have a jump server. This cost us in the end up to eight months. We had four solutions beforehand that were not successful. The fifth one was the jump server. This was the final one and is our solution at the moment.

In Germany, we have a higher security level. Therefore, it costs us by having a longer time to integrate. This is due to our requirements, not Worksoft. The reason why it took that long was on our side. We have a lot of IT departments. With security, social partners, and data privacy, there are a lot of requirements to fulfill.

After that, we agreed to have an automation manager on our team from Worksoft. He sat in Bratislava and all the open issues or questions sent to him, and he answered them, either directly or he contacted the support teams. Then, he assured that those Worksoft issues were solved. He also gave us hints how to use Worksoft, such as naming conventions and how to use it so you don't have a mess in the system.

After we had the automation manager for eight months, then he left. We decided to go with Cognizant, but that was also not successful. In the end, we decided to to have a Worksoft expert from Worksoft, if we need it. For example, we now need an for an expert for two weeks, who will tidy up our system. 

In the beginning, we had an integrator, Cognizant, who created a lot of scripts, but we stopped because we had a contract with them that they would install Worksoft, but it was not allowed. In the end, they just did some automated scripts for us. Our automation team in Bratislava was not satisfied with the quality in the end. We want to have a flexible style with quality, and this wasn't done by our Cognizant colleagues. Therefore, we decided to do it on our own.

From Cognizant, they were sold as SAP and Worksoft experts. Concerning Worksoft, I cannot say if they're expert or not. Concerning SAP, not all of them were experts, maybe one. However, this is always the same in this business. We also had cooperation with Worksoft. Concerning the flexibility of the scripts, we decided to work on our own.

We expect partners too be really good. Otherwise, the company does not see why we should pay for them.

Saving money and better quality, these are the benefits of Certify.

We have seen ROI. This was one of the goals from our top management when investing in automation. They want to see savings in the following year.

We did an RFP in 2016 and 2017. We found that Worksoft is easier to use because our business experts can do the tests. We didn't have to have IT experts, like with eCut.

If you use SAP, you can use this. It is easy.

I am really satisfied with the product. If I ask for support, I get support. I have direct contacts and every issue will be discussed. If we need something, they help us directly.

We did not automate our test maintenance. 

We don't have experience using it with apps and mobile testing, but are looking to add this to our portfolio in the future.

We are using Worksoft Certify to enable our stakeholders to do test automation on the UI level.

We use it for end-to-end testing of packaged applications. We are part of the internal IT department within our company. Most of the time we are using it on our own products. The products and systems that we get are usually preconfigured and prepackaged and we do additional testing, not just for the functionality for the coding that we add to the product, but also on the prepackaged solutions.

We use it for all kinds of applications. Our focus is the web area, including web-UI testing of modern applications. We have two tools in place: our own internal corporate test automation tool, and Worksoft Certify. The latter is a complementary tool, especially in the web area where there are some white spots for our corporate tool which it cannot cover. That was the main reason why we brought in another tool. And for that, it fits perfectly.

We are able to run our test phases faster. Once the scripts, the test cases, are ready and automated, not only are we able to check our systems or landscapes during the test phases, but we can proactively monitor our development and test systems. Proactive monitoring of our systems is very important for us and was not possible before because manual testing is just too time intensive.

Worksoft Certify helped us to increase time savings. We didn't start test automation in general with Worksoft Certify. We did automation before with our own tool, but it helped us to increase the coverage of test automation and to increase the time savings.

We had a success story with two teams. For the execution of the scripts, we had time savings of 82, 88, 95, and 90 percent. And for the speed, it was between nine and 21 times faster than manual execution.

It's not necessarily saving us money, but it's helping us to free up the capacities of our end users to work on other stuff. Instead of doing testing for two days, they can work on bug fixing, developing new features, etc. That person still gets the same paycheck at the end of the month, so it's not saving us money, but it increases the value of our products. It increases the quality of our products. The reason for that is we are not customer-facing. We are dealing with internal teams and internal products. We are not selling anything to the outside. We are with the internal IT department. For the development teams and the sales team or the consulting team it might be different. But we are not really going out, selling our products and getting the revenue for the company. This is done by other teams.

It's script-free, which is really important for our end users because we are usually dealing with colleagues who are not developers and who do not always have the technical background of developing and scripting. It's very useful that there is a nice UI and the tool is script-free.

It covers all of the technologies we need to cover.

And one big advantage of Worksoft Certify is its integration with SAP Solution Manager, the test suite of Solution Manager, with the certification. That is crucial for us since Solution Manager is our test management tool of choice.

There are a couple of small things, technically, that could be improved.

Features we have asked for include single sign-on. It's a bigger project to make sure that our end users do not have to store passwords, usernames, and the like, for the different tools we have. 

We are also working on an additional integration with another tool that we have in place for lights-out testing. That's ongoing at the moment.

Another idea we brought is that the definitions for the objects need to be automated. They need to be recognized automatically by Worksoft Certify instead of changing them back and forth manually. This is also something that Worksoft is currently working on. 

Updates, in general, is a topic that we are working on with Worksoft on a regular basis. For new products, for new UI technologies when they come out, the test-automation providers need to update their definitions to make sure that the objects are recognized correctly.

It is pretty stable. After upgrades, we usually don't experience any big issues. Of course, it's software, so here and there we find bugs, but nothing crazy, to be honest. The availability of the system is pretty good, almost 100 percent. I don't see an issue here.

At the moment we don't have any issues with scalability. We have about 300 end users working with Worksoft. On infrastructure, it's split. We have a WTS environment, a Citrix environment, for those colleagues who want to use the prepared environment. We have other colleagues who are using the client on their own machines, on their own laptops or desktops. The only thing that we saw at the beginning which we need to change in the future is that, due to the latency, we cannot use clients in the US, for example, while having the server in Germany.

If the latency is over a certain number of milliseconds then it is basically impossible to do automation. That was one of the main reasons why we set up the Citrix environment at the very beginning.

We are still in the phase within our company, or within IT services, of training and spreading the topic of test automation, overall. So our coverage, at the moment, is not the entire organization, it's only the IT department. Once we have done this - and it will take at least another year - we will see if we spread using Worksoft and our internal corporate tool as a combination, or tool ecosystem, further into the organization. But this is not our not our team's responsibility so it's not really in focus at the moment. We are pushing for test automation in our teams and there is still a huge demand for training and new teams coming into the topic of test automation.

In general, technical support is good. They are collaborative and responsive. The only thing I don't like - and this is the only complaint I usually have for Worksoft - is that the first-level support is not always the best for working on topics. We sometimes need to escalate to second-level support and then we know that we are getting a colleague who is aware of the issue and is not just playing for time.

We already reported this to Worksoft and asked them to find another way or to educate the first-level support or to make sure that the tickets go directly to second-level support if they come from us. The guys on our end who are reporting the issues sometimes know more than the first-level support.

When it comes to second-level support, we are happy. There, we know we will get the help that we need. The colleagues are responsive and very helpful and, from a quality perspective, they are very good.

In the beginning, there were some issues with the integration, it didn't work the way we wanted. We spent some time with the Worksoft team, with the support and engineering team, in adding some enhancements to adapt the solution to our three-tier Solution Manager landscape. But that worked very well.

We have a very good collaboration and relationship to Worksoft. For example, every two weeks we have calls with them. We'll provide feedback and they take it seriously. They usually provide us with updates, with enhancements, with new functionalities that we need. That's working pretty well.

We started with manual testing and then we started the test-automation initiative. We started with our internal corporate tool and then, as I mentioned earlier, we figured out that we could not cover everything with that. At the end of 2015, we started to check the market. We did some PoCs and we decided to go forward with Worksoft Certify.

There were a number of reasons we went with Worksoft Certify. The Worksoft team did a great job. They came to our headquarters and did the PoC, showing that the tool is suitable for our needs. They did another PoC with our operations colleagues who were running the regressions testing in Singapore. And then there were the technical requirements that we had. Worksoft Certify was able to cover all of them, some of which I have mentioned already: Being script-free, being fully integrated in Solution Manager, and being able to script in a modular way. And finally, the integration between our own internal tool and Worksoft Certify was also important.

The initial setup was pretty much straightforward. I cannot give you too many details because I did not take care of the implementation. But I know that it took us about two weeks to set up the whole infrastructure. It was not really difficult. And we had very good support from Worksoft, from the support and engineering team. They helped us in setting up the database, setting up the connections. It was not a big deal. In total, within two to four weeks, everything was working fine.

On our side, we had a couple of staff members involved in the implementation because our team is the application owner. We had to involve two more colleagues from the database team because we don't have all the authorization stuff, for the databases area, for the servers etc. In total we had about three staff members involved, but not full-time. It was about one work-week for each of them.

Maintenance is done by myself and one of my colleagues, with the help of our database and server/infrastructure team. We don't have authorizations for everything and we are not database experts. There are three or four staff members taking care of maintenance, as part of our job; it's not a full-time job, obviously.

Whenever we need to do a full upgrade, when we need to plan the downtime for the production system, we try to make it on the weekend. I also already recommended to Worksoft that it would be nice to have something like an offline update where the system can be upgraded or smaller changes and fixes can be included without having full downtime. For an upgrade we usually need two to three hours. Afterward, we do a bit of testing, so upgrading takes about half a day.

It was just one or two people from Worksoft and three guys on our side.

We get feedback from all areas that the return on investment is there. Not just regarding time savings, but also cost-reduction. The return on investment in one case was reached at something between five and six runs, which is pretty fast, especially in an Agile environment.

What is also very important for us here is the avoidance of human error during the execution of tests. Usually, if someone is sitting in front of a laptop and doing testing eight hours a day, he or she will make some mistakes. This does not happen with a tool. Another important factor for us is the availability for testing. Usually, it's pretty hard to plan a test phase to bring all the testers to the table and get the time blocked off for the test phases. For the tools, we just need the systems up and running and then it's a matter of minutes to set up the test plans and to run the tests.

At the moment we are rolling out Execution Manager.

At the time we decided to bring in Worksoft Certify, we looked into two other tools. The key difference was that Worksoft was script-free. That was not the case for other tools. And the full integration to Solution Manager was one of the key differentiators between the tools.

If you have done a market evaluation and have decided to go for Worksoft, my advice is to go for it. I would definitely recommend Worksoft Certify as a test automation tool.

The feedback that I get from our stakeholders is that the tool is pretty simple to use. What we usually do is a two-week training, not full time, where the total is about three to four business days, 20 to 25 work hours. From there, most of our colleagues can start working with the tool. Of course, they have questions later on, some difficulties when it gets into special activities. But overall, the tool is easy to use. It's generally found to be intuitive.

In terms of cutting test maintenance time with respect to the scripts, that has not happened. If you need to adapt your scripts, automated test scripts are much more complex and more effort-intensive than manual test cases. But this is the nature of the beast. It will happen with every tool. If a screen changes, if a system changes, then you have to adapt your script for manual testing. For a manual script, you just adapt a Word document or an Excel sheet or the like. But if the process flow changes, you have new windows, new options, then you have to adjust your script for each and every provider that you're selecting. The maintenance of scripts is something that I always discuss with my end users and should never be underestimated.

We are not using the Capture 2.0 feature at the moment. We are planning to use it in the future. But at the moment, due to the heavy workload on our plate, we haven't had the chance to look into and to roll it out. We are familiar with the concept of Capture and it's a very nice feature because it makes the collaboration between business and IT much easier, and business can be involved in test-automation topics and activities as well.

We have three roles in our environment. We have the key players, who are the project managers, the persons responsible for test automation overall in the respective teams.
Then we have the test automation engineers who are responsible for creating test scripts and to maintain them; sometimes they run them as well. And finally, we have the executors, the ones who are running the scripts, checking the details and, if something is not working fine, going back to the test-automation engineers and asking for support and help.

I rate Worksoft Certify at nine out of ten. I'm happy with the tool, I'm happy with our colleagues at Worksoft. We have a very good relationship, we can bring up everything. There isn't much I can complain about. I'm happy at the moment with Worksoft.