Worksoft Certify Reviews

We have been using Worksoft Certify for ten years.

We do automation of both SAP and other applications: One of them is our new grants management system. Originally, they wanted to do everything manually. Now, they highly rely on us to do testing in very short periods. 

We do web UI testing of modern applications. In our environment, we have SuccessFactors, plus SRM through Fiori. 

We also do performance testing and end-to-end testing of packaged applications, like SAP, InfoEd, and OnCore.

The most valuable features right now are the Capture capability and versioning. The Capture 2.0 has a lot of flexibility.

The easy of use and ease of integration of Worksoft Certify are very good.

We are looking for some enhancements on the Capture 2.0 tool. This would give us the ability to control it directly, like we could with Capture 1.0. Right now, Capture 2.0 doesn't really work for our Business Analysts. Because they don't work with it all the time, it's a little complicated for them to stay up to speed on it. With Capture 1.0, we wrote a wrapper to make it easier for them to use, but we can't use that wrapper with Capture 2.0. So, if Capture 2.0 gets enhanced, we'll start using it. For an enhancement, we want to be able to start and stop recording through an API. Then, we want to see how many steps have been recorded through the API.

We do a lot of test maintenance because they are constantly changing the applications. This is one of our biggest problems that are constantly making changes and switching products. For example, we used to use the Supply Relationship Manager. Now, all those tests that we build there will be replaced when we go to Ariba. All the old SAP GUI stuff for HCM, when we got SuccessFactors was thrown out, then had to be redone with SuccessFactors.

It gives you the ability to have a more structured environment for tests. It is not just recording of key strokes. It is more systematized, more like a programming language. That is the biggest advantage for us. Because of its consistency, once the developers know one skill set, they can use it to automate any application.

It scales well. We have eight offshore people and four developers in New York. We can add developers as needed. With the offshore team, we've have at least 20 people trained on Worksoft, but eight people actively on the team now.

We have about a 1000 active users with about 5000 users total. However, this includes all of the employees and their self-service.

The technical support is very good. They have always been a great help. With a lot of companies, you can't even really talk to the developers that someones provides you. With Worksoft, you can get a hold of the developers and work with them. It is all very timely.

The initial setup was very straightforward. We just build the database and installed the client application on the system. There was nothing out of the ordinary about it. It is not like setting up a lot of the IBM products or the Quality Manager, where there were so many different modules. There was just one module to set up.

During our yearly upgrades, we have now gotten them down to ten days or less. We have Worksoft run all our integration tests, where it used to take probably six weeks to do that manually.

As far as maintenance testing, it has saved us time. We find problems a lot of the time that they aren't aware of. This is because we run the tests even though they don't ask us.

We evaluated Micro Focus QuickTest Professional (QTP).

Properly staff the testing team before they attempt to do automation. Be aware that this will not be a one-time overnight process. We tried to automate everything in two months with eight people, and it was impossible.

We have been using it for so many years that we are really very happy with it.

We will be converting to Ariba for purchasing, so that will have to be automated.

Worksoft Certify is being used to run automated weekly regression tests across some of our primary SAP systems in line with our Change and Release management strategy. These tests run every weekend without fail. The results are reviewed on every Monday morning to check for failures and to analyse if any failures are associated with the changes scheduled to be transported to the production environment that week. Failures (if any) are fixed and the tests re-run before transporting the associated changes into the Production environment.

We also utilize it for projects that need extensive business-user testing and functional testing. There can be testing requirements which come at short notice which can take three to four weeks of manual testing effort. By using Certify, we have been able to bring timescales down to a few hours of automated testing effort.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems which will demand extensive regression testing. 

We have never had any systematic regression testing regime in the organisation. This has helped in building an automation framework across our SAP application landscape, thereby introducing mandatory regression testing across all our key systems and improving the overall quality across our production systems.

From an audit perspective, results generated from Certify (BPP reports) provides detailed test evidence which is also being utilized for internal training purposes/training guides, etc. The BPP reports also provide details on failures along with screenshots.

We have a variety of complex systems in our landscape, one of them being the Openlink Endur which is a commodity trading and risk management system. We are currently building an automated regression test suite to support application testing for Endur.

Our weekend regression tests are performed in 'lights-out; mode. Tests are scheduled to run at a certain time over the weekend using the Execution manager functionality. Usage of Certify has also prevented some major defects going into Production and we have seen significant savings in all manual testing activities as the business users/functional teams are getting more time to perform 'value- adding' activities.

Post our recent upgrade to Solution Manager 7.2, we are currently in the process of implementing the Test suite functionality and the integration of the same with Certify. We expect Solution manager to be the single source of truth bringing out all the results from Certify which is going to be extremely beneficial from an audit perspective. We have already implemented the integration of Certify with HP ALM in our landscape.

Moreover, we have this reusable asset now which can be run frequently to support all our projects and change requests across our legacy SAP systems. Even last-minute testing requests are being accommodated by utilising the automated regression suite without any dependency on business users/functional users for their efforts. We use it across the multiple projects which need immediate assistance and for our weekly regression cycles. To give an example of a recent project which was a major platform migration from a Data Centre in Asia to Europe which needed extensive Disaster recovery testing and Functional testing/User acceptance testing. The initial testing estimate was approximately five to six weeks, however with the use of Certify we could do extensive testing in less than three hours saving many weeks of manual testing effort.

It is very user-friendly with an appealing UI, unlike a lot of other automation tools that we have evaluated. With sufficient training and adoption of best practices, the tool will certainly help organisations to successfully implement an automated testing framework and eliminate manual testing activities.

The fact that it can be used across SAP and non-SAP applications (including web-based apps like Web Dynpro) is a big advantage for us because we have a variety of SAP and non-SAP applications across the Johnson Matthey IT landscape. Being a 200-year-old organisation, our variety of legacy systems have a lot to benefit from the use of automated testing.

Certify has many interesting features, e.g.: 'PRIMO' which is the image recognition functionality is a life saver in instances where Certify standard functionality cannot identify and learn objects within certain legacy applications.

Regarding end-to-end testing of packaged applications, Certify is primarily used across our SAP application landscape and the Openlink Endur (commodity trading and risk management system). We hope to realise more benefits by implementing Certify across our wider application landscape over the next few months.

We have been using the Capture feature, although not the latest version, the initial version, for process captures was used to create our test designs. It has been a life saver in many instances, without the need to spend any extra effort to create test designs and captures. The test steps get captured in the background which generates an XML file which can be easily imported into Certify, creating the basic test structure which can be improvised/modified to make it a repeatable reusable test. In terms of the amount of time it takes users to create documentation automation using this feature, it is the same amount one would spend to do a manual test. While a person is performing a manual test, Captures are automatically generated in the background. We have used it extensively to build our test designs.

We have requested for some minor new features which Worksoft is considering.

The PRIMO image recognition functionality has room for improvement, especially around its ability to work with java interfaces, Execution manager scheduling, etc. as we have observed. As we explore more of our legacy systems, I am certain there will be a need to use more of the PRIMO features to learn the objects.

Overall from a SAP perspective, it works almost seamlessly.

One to three years.

The version of Certify that we are using has been mostly stable and we have rarely encountered any problems. Our weekend regression test failures are often associated with environmental/system performance issues and not related to the stability of Certify. I have been happy with the overall performance of Certify and how it has helped to optimise our tests.

I am confident that Certify can scale to fit our automation testing needs as we expand the current automation testing framework across the wider Johnson Matthey application landscape. We are also exploring options to identify potential areas where Certify can help support mass data uploads, etc. to benefit other teams in their day to day operations.

We have several concurrent users accessing Certify in our environment, primarily automation engineers, test engineers and tech managers.

Our interactions with technical support has not been the best always and there is room for improvement especially with respect to the time taken to respond to cases. However, with the right contacts and reasonable escalations we have always managed to get quick attention on our issues.

We have always been heavily reliant on manual testing and as a result, regression testing was not systematic and we could never think of implementing frequent weekly regression test cycles which was challenging. We decided to go ahead with automated testing and use Certify because:

1. Manual regression testing takes a lot of time and resourcing is always a challenge.
2. Regression testing not being systematic, the quality was very difficult to measure as we did not have a standard set of manual regression test scripts/sufficient documentation.
3. There was a delay in our time-to-market because all the testing was being done manually and there was no way we could accommodate frequent, weekly, regression test cycles. That meant high business risk, that we would have more defects in the production environment/ more associated costs.

We had all these challenges and we started exploring options to mitigate these risks and automation was identified as the way forward, nearly two years ago. We evaluated various automation tools in the market. It was critical that we had to identify a strategic tool which would cater to our SAP and non-SAP application landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

We went through a lot of initial challenges, mostly around internal resourcing issues. Looking back, I am happy to say that we could overcome these challenges and have managed to successfully implement an automation framework using Certify.

Early in 2017, we decided to go ahead with Worksoft Certify post evaluation of multiple automation tools. Our initial engagements with Worksoft consisted of several onsite workshops to explore the tool in detail along with technical feasibility assessments across our application landscape. These engagements were extremely beneficial and it gave us the overall confidence to adopt Worksoft Certify as our strategic test automation tool.  

We did a pilot implementation with Worksoft to see if we could take this ahead on a large scale before embarking on the major project to build the automated tests. Some key processes across our critical SAP systems were identified as candidates for this exercise. Test designs were created with support from the functional teams and taken ahead for automation build with Senior Worksoft consultants and our internal resources. This 7-week Automation Roadmap Engagement exercise was extremely successful and we learned a lot of lessons from it which helped us plan the next big phase of the automation roll out. It gave us overall confidence across the functional and management teams which subsequently led to securing the appropriate budget, etc.

One of the biggest lessons learned from this engagement was around the ways to structure our teams. This led to us going ahead with a Managed Services model with Worksoft. We have an offshore based Worksoft Automation Services Factory team who helps build our automated tests. The team can scale up/down based on our automation forecasts.

The automation deployment is still ongoing. The initial phase was completed across a five-month span. Currently we are rolling out the second phase of the automation build focusing primarily on our global Unify solution and the Openlink Endur application.

Regarding implementation strategy, we followed an agile two-week sprint approach. Our functional teams continuously created test designs and these were fed to the Automation Factory every two weeks, who in turn developed the automated tests. This was the most practical model, which worked well in our environment.

At its maximum capacity, we have had approx. 10 to 12 automation engineers in the Factory team. Our functional teams are spread across multiple global locations and we had between 3 to 6 resources working on test designs liaising with the business users as required.

From a script maintenance perspective, we spend an average of 4 to 5 hours every week with the current asset of nearly 800+ tests.

We have always worked directly with Worksoft, along with support from our internal resources. Worksoft has been delivering excellent services through their managed services model.

We have a res-usable re-runnable asset built which is saving a lot of time across the functional teams/business user community.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems, which will demand extensive regression testing. 

It is expensive compared to some of the other automation tools in the market. However, the benefits and ROI has proved that it has been a good investment.

We have concurrent licenses.

SAP TAO and Micro Focus UFT.

It was critical to identify a strategic tool which would cater to the testing requirements across our SAP and non-SAP application (including web based apps like Web Dynpro) landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

It is a great product and we have not seen anything which cannot be automated till date in our application landscape.

It is important to do sufficient technical feasibility assessments before deciding to go ahead with Certify and equally important to determine the best implementation approach which will work for your organisation. Functional teams/business users' buy in is critical as the test designs cannot be created without their continued support. Adoption of best practices around naming conventions/folder structures etc. will help in easy overall maintenance of the test assets, which will also help with the generation of development and execution dashboards/overall reporting.

I would rate Certify at eight out of ten. Worksoft has always been very supportive and responsive to our needs and this has certainly helped us achieve our initial milestones successfully. I am extremely proud of what has been achieved so far and looking forward to expanding the automation framework across our wider IT application landscape over the months ahead.

The primary use case is to automate their SAP and Web applications. We use Worksoft Certify for end-to-end regression and integration testing across our systems.

It helps us to implement automation testing as part of most projects, so the need for manual testing can be reduced. This really accelerates the testing process as a whole. Before, where it could take ten days to test a project, now it takes only one or two days to do the complete testing. So, it helps us to reduce our testing timelines.

We no longer need ten people sitting and manually testing something. We can just have one person running the entire regression automation testing suite, and this has saved us dollars.

I have found Capture 2.0 helpful compared to Capture 1.0. It allows you to have access to all the screenshots when you use it so you see what steps you are capturing, and if you are capturing it right or not. It is very user-friendly.

It is easy to use and learn the application. For example, I have an intern who joined me three months ago. Today, she delivers the same number of scripts as my experienced developers with great quality.

One feature that could be added to Capture 2.0 is generating a PDF file from your capture, so you can see your screenshots and steps. This will really help teams leverage the documents generated as part of requirement/training.

Right now, when we do regression testing, we manually have to generate all the reports and populate all the results in HP ALM. We really are looking for a solution to have send all the results to HP ALM once Execution Manager completes the execution, then automatically logs them.

Our offshore teams experience a lag/delay when using the Worksoft interface. As of now we use VPN and Remote Desktop to help us with this issues, it was be great to see how much Certify 11 has improved in terms to offshore accessibility.

Every time there is a new release of Worksoft, they present it in a conference. However, there is no training document nor one point of solution where I one see what new changes/feature have been implemented, like a portal. If I don't know how to use a feature, there is no training nor documentation available. When you reach out for support, it takes time for them to research it and get back to us.

I would like more use cases or at least a weekly email update to all the customers saying, "These are new features which have been included in the last week." That would really help.

Worksoft with SAP is pretty stable. 

When it comes to the web, you need a lot of extra effort in making sure that the tests are maintained, but that is the nature of the web application. You have stuff that is changing all the time, so you have to ensure that you maintain your tests regularly.

We have learned the scalability part over time. We were very successful with SAP, but not very successful with the web. Over the period of about two years, we were able to prove to our company that we can use Worksoft for different type of applications. While there is a learning curve, it is all about trying things out and failing few times before you get a success.

I would rate the technical support as a seven out of ten. There is a delay in time zones when we reach out to them. The response is not as quick as we expect it to be with the other solutions that we have. There is definitely a delay in timing, as the speed is lacking.

The initial setup was easy. We had all the information and the requirements that we needed to do the prep work before doing the implementation.

We have Worksoft help us each time that we do an upgrade or implementation. Our  upgrades have gone smoothly.

An example of saving time and dollars: We had project going on that used to require manual testing. The first time that they did manual regression testing, we had a group of about 15 testers who sat in our office for a period of two weeks to do the testing. Now, when we have to regression testing with automation scripts ready in our Worksoft Certify, we do it in a day or two. 

We looked at quite a few vendors and automation tools in the market. We chose Worksoft because of the ease of use and mentorship support they provide in making sure we were successful.

For a new customer who is to implement Worksoft Certify, I would suggest 'Start the right way'. Have a Worksoft mentor come in and help you with your automation journey specific to your organization so you can have expert support until you become successful with it. Once you are successful, you'll know what to do. E.g., we had a team of interns who got trained and they tried to work with it, but it did not work. Then, we had Worksoft help us (after two years), and it worked. 

My team and I do the regression testing. We are a team of three to four people. We are not working on just one project, we are working on five or six different projects.

What's next, well I hope we are able to present that next success story in next conference ;)

We have an SAP environment, so we use Worksoft for SAP and the ecosystem around SAP. Most of the use cases are related to SAP products or interfaces and the applications that are interacting with SAP.

We use it for test automation. We are basically using it for regression testing, especially for our releases. For example, in the big SAP systems, when we have support package upgrades or bigger function releases, we use end-to-end test automation to ensure that the changes are not impacting the processes in the system. With this test effort, we can make sure that the releases are running without any issues in the production systems.

We started using it around six years ago with an on-prem installation, and we had a pretty good experience with that. The way we are using the software is that we have installed it on our terminal server so that not every tester has to install it on his own machine. Having this terminal server environment is allowing us to really stick to specific standards in terms of how the software will be used and in which sequence updates will be distributed on the server. It also helps in terms of the connectivity to the systems that we require for test automation. It makes it quite easy for people to concentrate on developing tests and not on the environment. 

We are running version 12.0, and 2006.77 is the patch level. 

In terms of its web UI testing abilities for testing modern applications, such as SAP Fiori, we started implementing a project two years ago where we developed a logistic layer and a finance layer, which all the future SAP systems of the headquarter divisions will be using. For that project, we had introduced Worksoft for automated testing. We are quite heavily using Worksoft in that area. We have all of our core functionality in that area automated, and we had a really good experience with Fiori. 

Worksoft has these so-called configuration files that you can get for different applications to define the maps. We are also using ServiceNow or Pega for Workforce management. For both applications, you can get so-called configuration files from Worksoft, and with these configuration files, Worksoft can very easily identify the objects. So, you don't need to learn Worksoft from scratch, but you can really build on the foundation of already-existing definitions coming from Worksoft. 

It provides codeless end-to-end process automation across packaged applications. It does not have the approach of writing scripts or having a scripting language for the logic. It is pretty easy to adopt. It is helping us in general because you don't need a developer or a technical guy for building these scripts. People in the business organization can really design their own scripts without heavy IT support. Normally, we just teach testers how to work with Certify in general for a couple of hours. If they have understood the basic patterns in terms of how to find specific commands, how to really work with these conditions, and how to work with varietals, after a day or two, they are able to work with this solution. They might sometimes ask where to find specific things, but because Worksoft also provides master content with a lot of examples, they can deal with it from there. In our company, we have an approach that all people work on the same project. This means that they are also sharing their scripts internally so they can read and steal from others. We also have a concept that for every SAP system, there should be one test architect who is knowledgeable. He is a key user, and he drives the effort to bring knowledge to people. 

It definitely reduces the time you spend on test maintenance. The debug feature, the recognition feature, and the decoupling of scripts and maps are really saving time. Imagine having an error at step 850 in a test script that has 1,000 test steps, and these 850 steps have taken you an hour for execution. In such a case, you have to repeat the entire test because you don't have the possibility to go back to certain steps. Every time, you will lose an hour or two in maintenance. Having these features makes it pretty effective and efficient, but it is hard to say the exact time because you don't know how often your scripts are breaking because of updates. It also depends on the number of scripts. We also have to see the number of saved hours in relation to other tools. So, if you're comparing it with an open-source test automation tool like Selenium, it might be saving you more time, but that might not be the case if you're comparing it with Micro Focus or Tosca. 

It has definitely enabled us to scale up our testing. When you use automated test scripts for test cases, your testers are released from that testing time, and they can concentrate on further testing. The way we are introducing test automation in our organization is that we say, "Okay. This dummy type of testing can be done by a robot such as Certify," and then our testers, who are hopefully more intelligent than the machine, can concentrate more on the individual tests. You cannot really automate all the test cases, and it allows our testers to concentrate on the individual test cases.

Worksoft Certify works well for creating test scripts. As compared to other tools for test automation, what is very good in this tool is the ability to implement logic into the scripts without coding and learning a complex script language. It is comparable to defining formulas in Excel. It is pretty easy to learn how to make your scripts more intelligent and more flexible as per the situation.

The decoupling of the test scripts from the data and the application is also a nice feature. When you are creating test scripts, for example, for a web application, you have to learn about Worksoft and how the controls of a screen can be interpreted by Worksoft. For that purpose, you create so-called maps. These maps are loosely coupled to your scripts, which means if the application is changed, the control will be changed from an identifier. You don't need to rework the entire script. You only need to do these adjustments in the map, and then you can automatically reuse the scripts. So, it is really a smart move to have the decoupling of scripts, maps, and data. 

Its debugging functionality is pretty powerful as compared to other tools. Recognizing the errors sometimes could be challenging. When the debug function, for debugging your scripts, runs on an error, it can stop at that error and identify the elements that may have not been recognized. It can then update the definition to recognize the object. It then repeats the step again so that you have a so-called execution pointer, which you can then use for your debugging. 

Certify is integrated with Solution Manager, but this integration could be easier. 

Overall, in terms of how it is working, I find it pretty clever in all the areas. There are only tiny things. For example, to log into Certify, you have to put in your username and password. In version 12, they changed it, and the password is no longer stored. So, you have to enter it every time you log in. Similarly, there should be a way to store the layout of tables in Certify. You can adjust your tables, but when you close Certify, if I recall correctly, the layout of the table is not stored automatically. So, you have to adjust it every time. I'm, however, not quite certain about it. 

These are tiny things that they can improve, but compared to the whole feature list of Certify, they are not so important.

I have been using this solution for around six years.

We regularly update the software when we see that there are new features available or if there are fixes in certain areas. In general, the Certify software is pretty stable. Based on our experience, there is no need to import patches every month or on very short notice. We normally plan for once a year version update.

In our S4 project, we had the need to develop automated testing for Excel-based solutions. We needed to test the business planning functionality that was running in Excel from SAP. It is quite challenging to build automated test scripts in desktop applications like Excel, but we got quite good support from the offshore team of Worksoft. We had a talk with an engagement manager from Worksoft, and then someone from India came to Lisbon, Portugal, and they all worked together. Our team quite quickly learned how to handle the challenges in that area. So, it is not only about the tool; it is also about the support you are getting from Worksoft. Their support was quite impressive.

It is pretty powerful as compared to other tools. We developed our own tool, and we have also compared it with Micro Focus. We have some knowledge of QTP from HP and Tosca Tricentis. From my perspective, especially when it comes to debugging and also object recognition, Worksoft may be one or two years ahead as compared to the other tools.

When we started with this solution, we had an engagement program. We had a consultant from Worksoft for 20 or 30 days on demand. It was an engagement contract that we had signed while acquiring the licenses. We had two or three onsite sessions. This consultant was here in Berlin with me and helped with the installation and documentation. This engagement really should be seen as enablement. It was not that the consultant did everything and then handed over the documentation. These sessions were more like hands-on sessions, which means our administrators understood how to install the software, how to configure the software, and how to make connections between different applications, especially with the database. They also understood how to make sure that our security regulations are met because there were some problems there. After we had documented everything, the consultant did his job with other clients, and we continued to handle the software on our own. We are deploying patches these days without any support from Worksoft because we simply learned how to do it.

Its initial setup is complex. There is the client part and the database part that you have to install. The client installation is pretty easy and straightforward, and you just have to click the Next button. For the database part, there are SQL Server scripts that need to be executed on the database server. It is pretty simple. You have scripts running on the database, and typically, they run without errors. In all these years, we had problems with the upgrade only twice. We have a QA environment where we typically test the upgrades. We had an error because a column was missing in the table. We raised a ticket, and someone from Worksoft helped us. We learned how to handle it and did the same on the production system without any support. 

If you give me a system, a database server, and maybe a terminal server and we have to install both parts, the database part can be done in one or two hours, which includes preparation time, execution time, and post-installation time. Overall, it would take a day because the database also requires some time for installation. If you are simply differentiating between the effort and the duration, in terms of duration, the database would take a day. In terms of effort, it would take one or two hours. The client part also takes one to two hours, depending on the resource you are using. After that, you only need to do the configuration to connect to the license server and the database. If you know what to do, it would be up and running in a maximum of two hours. We are not really talking about a complex SAP system. It is simply a test automation tool.

We have seen an ROI. In the end, it is money and time. You save time for the testing, and you also save time in making corrections. If you don't have such high-quality testing, you will end up with errors in the production system. You will also have some interruptions in the daily business in your SAP systems. That's one aspect of the return on investment. The easiest way to calculate the ROI is in terms of the effort that you are reducing for testing.

I can only judge based on the situation that we had around six years ago when we did the tool evaluation. Worksoft was not the cheapest, but it provided the value. For 25 concurrent licenses, we paid more than €400,000, so it was not cheap. In the end, if you see how much time you are saving and compare it with others, its price is okay. We had also compared its cost with the licensing costs for HP and Tricentis, and they were at another level.

Now, as we have already booked the licenses, we only have to pay an annual maintenance fee, which is 70%, and that is okay.

The biggest change was not really the tool. There is the saying, "A fool with a tool is still a fool." That's pretty much true. When you are starting with test automation, you basically have to understand the concepts behind test automation, and you have to learn how the robot does the testing. Normally, your testers are reacting, and they are pretty flexible. For example, if they recognize that something is blocking a storage location, they free up the storage location and continue. If you are doing the same with an automated test script, this needs to be implemented in the test script or logic. This is pretty much the difference. So, you need to be very precise in knowing the circumstances or issues that the tool might come across during a test. You also have to have a big focus on the test data. That's because if someone changes your master data, your test scripts will fail, and you won't be able to differentiate whether the error is on the system side or the data side. 

You also need to think about how you are building your end-to-end tests. In the past, most of our tests were in the area of functional tests, but for the dependencies between the different functions, we really had to concentrate on end-to-end testing. This is pretty much the challenge when people from different organizations have to work together. There must be someone from the purchasing team and the finance team to negotiate on specific test cases and test data, which really takes time. With Certify, you have a tool with which you can concentrate on the content and the logic of your end-to-end scripts, and you don't need to spend so much time handling the tool. A good piece of advice for someone who would like to use Certify is that do not concentrate so much on the tool. You should concentrate more on the concepts and circumstances, such as how to ensure the stability of your systems and data. Are you going to introduce a pre-prod system, an isolated system, or an environment? That is more challenging than the tool.

We are using the Capture feature to capture a sequence of our test. Once this sequence is recorded in Capture, we then transfer it to Certify and continue the development there. The Capture feature is kind of a movie that you create. This movie is transferred to the Certify tool, and you can use a feature called BBP to transfer your test scripts into multiple formats. You can transfer it to PDF or Word format. You can show the process documentation with screenshots in a Word document, but in our company, we are very much standardized and formalized. So, this kind of process documentation is not sufficient. We can use it for simple documentation, for example, for discussing change requests for an SAP system, but for comprehensive detailed documentation, we have tools in place.

We have different tools in our company for RPA. RPA is not really in the area of Worksoft. I know that some of the organizations that are using Worksoft Certify for automation are also using it for RPA, but this is more of an exceptional case.

I would rate Worksoft Certify a nine out of 10. I'm pretty confident of and satisfied with this tool, but there is always room for improvement.

It serves as our primary automation tool, specifically utilized for testing purposes. The tool predominantly finds application in functional testing and integration testing for our customers.

The tool itself is highly effective, especially when it comes to comprehensibility for newcomers. Even during the initial learning phase, I found it remarkably user-friendly. It facilitates quick onboarding and training of new resources. It offers features for building automation scripts, such as search and certify capture. A notable advantage is that it doesn't rely on other tools like Micro Focus UFT or Micro Focus ALM for script execution. Worksoft can run independently without the need for support from Microsoft Office, although the option to use it for maintaining data exists. The tool also includes debugging features and comprehensive reporting capabilities, generating PDF reports for easy analysis.

The primary area for improvement is the support service. The support process can be time-consuming, with tickets sometimes remaining unresolved even after considerable follow-up. Also, it doesn't offer a training or trial version for users to explore the tool before committing. Enhancing direct connectivity between radio and Worksoft Certify would streamline and improve efficiency. Although there are YouTube videos for specific features, the Worksoft community lacks corresponding content. This has been a common concern among users who find information on YouTube but struggle to locate it on their community portal.

I have been working with it for over eight years now.

I would rate its stability capabilities eight out of ten. Stability concerns are significant in automation, especially when dealing with new applications or features. Challenges arise in assessing the feasibility of automation without adequate support.

I would rate the scalability ten out of ten. At the moment, we are involved in four distinct projects out of the overall portfolio, which consists of around thirty projects. The focus is primarily on high-scale business initiatives.

While having a partner with dedicated support can help in critical situations, as a customer, the overall support experience might not always meet expectations. It often involves multiple calls and explanations to various teams across different locations, making it a bit cumbersome. I would rate it six out of ten.

Neutral

The initial setup was straightforward. I would rate it nine out of ten.

The deployment typically takes one to two weeks. The first step involves setting up the database, determining the server specifications, and installing the servers based on the specific requirements and data activities of the project.

While I acknowledge that it's a good tool, the challenges with support are notable. Despite the tool's excellence and the introduction of new features, the ground-level implementation should align with the promised benefits. In comparison to other tools in the market, some offer superior support and features. However, it's worth noting that it remains one of the best tools for learning, especially for users who might not have many alternatives. Overall, I would rate it ten out of ten.

We are using Worksoft Certify to enable our stakeholders to do test automation on the UI level.

We use it for end-to-end testing of packaged applications. We are part of the internal IT department within our company. Most of the time we are using it on our own products. The products and systems that we get are usually preconfigured and prepackaged and we do additional testing, not just for the functionality for the coding that we add to the product, but also on the prepackaged solutions.

We use it for all kinds of applications. Our focus is the web area, including web-UI testing of modern applications. We have two tools in place: our own internal corporate test automation tool, and Worksoft Certify. The latter is a complementary tool, especially in the web area where there are some white spots for our corporate tool which it cannot cover. That was the main reason why we brought in another tool. And for that, it fits perfectly.

We are able to run our test phases faster. Once the scripts, the test cases, are ready and automated, not only are we able to check our systems or landscapes during the test phases, but we can proactively monitor our development and test systems. Proactive monitoring of our systems is very important for us and was not possible before because manual testing is just too time intensive.

Worksoft Certify helped us to increase time savings. We didn't start test automation in general with Worksoft Certify. We did automation before with our own tool, but it helped us to increase the coverage of test automation and to increase the time savings.

We had a success story with two teams. For the execution of the scripts, we had time savings of 82, 88, 95, and 90 percent. And for the speed, it was between nine and 21 times faster than manual execution.

It's not necessarily saving us money, but it's helping us to free up the capacities of our end users to work on other stuff. Instead of doing testing for two days, they can work on bug fixing, developing new features, etc. That person still gets the same paycheck at the end of the month, so it's not saving us money, but it increases the value of our products. It increases the quality of our products. The reason for that is we are not customer-facing. We are dealing with internal teams and internal products. We are not selling anything to the outside. We are with the internal IT department. For the development teams and the sales team or the consulting team it might be different. But we are not really going out, selling our products and getting the revenue for the company. This is done by other teams.

It's script-free, which is really important for our end users because we are usually dealing with colleagues who are not developers and who do not always have the technical background of developing and scripting. It's very useful that there is a nice UI and the tool is script-free.

It covers all of the technologies we need to cover.

And one big advantage of Worksoft Certify is its integration with SAP Solution Manager, the test suite of Solution Manager, with the certification. That is crucial for us since Solution Manager is our test management tool of choice.

There are a couple of small things, technically, that could be improved.

Features we have asked for include single sign-on. It's a bigger project to make sure that our end users do not have to store passwords, usernames, and the like, for the different tools we have. 

We are also working on an additional integration with another tool that we have in place for lights-out testing. That's ongoing at the moment.

Another idea we brought is that the definitions for the objects need to be automated. They need to be recognized automatically by Worksoft Certify instead of changing them back and forth manually. This is also something that Worksoft is currently working on. 

Updates, in general, is a topic that we are working on with Worksoft on a regular basis. For new products, for new UI technologies when they come out, the test-automation providers need to update their definitions to make sure that the objects are recognized correctly.

It is pretty stable. After upgrades, we usually don't experience any big issues. Of course, it's software, so here and there we find bugs, but nothing crazy, to be honest. The availability of the system is pretty good, almost 100 percent. I don't see an issue here.

At the moment we don't have any issues with scalability. We have about 300 end users working with Worksoft. On infrastructure, it's split. We have a WTS environment, a Citrix environment, for those colleagues who want to use the prepared environment. We have other colleagues who are using the client on their own machines, on their own laptops or desktops. The only thing that we saw at the beginning which we need to change in the future is that, due to the latency, we cannot use clients in the US, for example, while having the server in Germany.

If the latency is over a certain number of milliseconds then it is basically impossible to do automation. That was one of the main reasons why we set up the Citrix environment at the very beginning.

We are still in the phase within our company, or within IT services, of training and spreading the topic of test automation, overall. So our coverage, at the moment, is not the entire organization, it's only the IT department. Once we have done this - and it will take at least another year - we will see if we spread using Worksoft and our internal corporate tool as a combination, or tool ecosystem, further into the organization. But this is not our not our team's responsibility so it's not really in focus at the moment. We are pushing for test automation in our teams and there is still a huge demand for training and new teams coming into the topic of test automation.

In general, technical support is good. They are collaborative and responsive. The only thing I don't like - and this is the only complaint I usually have for Worksoft - is that the first-level support is not always the best for working on topics. We sometimes need to escalate to second-level support and then we know that we are getting a colleague who is aware of the issue and is not just playing for time.

We already reported this to Worksoft and asked them to find another way or to educate the first-level support or to make sure that the tickets go directly to second-level support if they come from us. The guys on our end who are reporting the issues sometimes know more than the first-level support.

When it comes to second-level support, we are happy. There, we know we will get the help that we need. The colleagues are responsive and very helpful and, from a quality perspective, they are very good.

In the beginning, there were some issues with the integration, it didn't work the way we wanted. We spent some time with the Worksoft team, with the support and engineering team, in adding some enhancements to adapt the solution to our three-tier Solution Manager landscape. But that worked very well.

We have a very good collaboration and relationship to Worksoft. For example, every two weeks we have calls with them. We'll provide feedback and they take it seriously. They usually provide us with updates, with enhancements, with new functionalities that we need. That's working pretty well.

We started with manual testing and then we started the test-automation initiative. We started with our internal corporate tool and then, as I mentioned earlier, we figured out that we could not cover everything with that. At the end of 2015, we started to check the market. We did some PoCs and we decided to go forward with Worksoft Certify.

There were a number of reasons we went with Worksoft Certify. The Worksoft team did a great job. They came to our headquarters and did the PoC, showing that the tool is suitable for our needs. They did another PoC with our operations colleagues who were running the regressions testing in Singapore. And then there were the technical requirements that we had. Worksoft Certify was able to cover all of them, some of which I have mentioned already: Being script-free, being fully integrated in Solution Manager, and being able to script in a modular way. And finally, the integration between our own internal tool and Worksoft Certify was also important.

The initial setup was pretty much straightforward. I cannot give you too many details because I did not take care of the implementation. But I know that it took us about two weeks to set up the whole infrastructure. It was not really difficult. And we had very good support from Worksoft, from the support and engineering team. They helped us in setting up the database, setting up the connections. It was not a big deal. In total, within two to four weeks, everything was working fine.

On our side, we had a couple of staff members involved in the implementation because our team is the application owner. We had to involve two more colleagues from the database team because we don't have all the authorization stuff, for the databases area, for the servers etc. In total we had about three staff members involved, but not full-time. It was about one work-week for each of them.

Maintenance is done by myself and one of my colleagues, with the help of our database and server/infrastructure team. We don't have authorizations for everything and we are not database experts. There are three or four staff members taking care of maintenance, as part of our job; it's not a full-time job, obviously.

Whenever we need to do a full upgrade, when we need to plan the downtime for the production system, we try to make it on the weekend. I also already recommended to Worksoft that it would be nice to have something like an offline update where the system can be upgraded or smaller changes and fixes can be included without having full downtime. For an upgrade we usually need two to three hours. Afterward, we do a bit of testing, so upgrading takes about half a day.

It was just one or two people from Worksoft and three guys on our side.

We get feedback from all areas that the return on investment is there. Not just regarding time savings, but also cost-reduction. The return on investment in one case was reached at something between five and six runs, which is pretty fast, especially in an Agile environment.

What is also very important for us here is the avoidance of human error during the execution of tests. Usually, if someone is sitting in front of a laptop and doing testing eight hours a day, he or she will make some mistakes. This does not happen with a tool. Another important factor for us is the availability for testing. Usually, it's pretty hard to plan a test phase to bring all the testers to the table and get the time blocked off for the test phases. For the tools, we just need the systems up and running and then it's a matter of minutes to set up the test plans and to run the tests.

At the moment we are rolling out Execution Manager.

At the time we decided to bring in Worksoft Certify, we looked into two other tools. The key difference was that Worksoft was script-free. That was not the case for other tools. And the full integration to Solution Manager was one of the key differentiators between the tools.

If you have done a market evaluation and have decided to go for Worksoft, my advice is to go for it. I would definitely recommend Worksoft Certify as a test automation tool.

The feedback that I get from our stakeholders is that the tool is pretty simple to use. What we usually do is a two-week training, not full time, where the total is about three to four business days, 20 to 25 work hours. From there, most of our colleagues can start working with the tool. Of course, they have questions later on, some difficulties when it gets into special activities. But overall, the tool is easy to use. It's generally found to be intuitive.

In terms of cutting test maintenance time with respect to the scripts, that has not happened. If you need to adapt your scripts, automated test scripts are much more complex and more effort-intensive than manual test cases. But this is the nature of the beast. It will happen with every tool. If a screen changes, if a system changes, then you have to adapt your script for manual testing. For a manual script, you just adapt a Word document or an Excel sheet or the like. But if the process flow changes, you have new windows, new options, then you have to adjust your script for each and every provider that you're selecting. The maintenance of scripts is something that I always discuss with my end users and should never be underestimated.

We are not using the Capture 2.0 feature at the moment. We are planning to use it in the future. But at the moment, due to the heavy workload on our plate, we haven't had the chance to look into and to roll it out. We are familiar with the concept of Capture and it's a very nice feature because it makes the collaboration between business and IT much easier, and business can be involved in test-automation topics and activities as well.

We have three roles in our environment. We have the key players, who are the project managers, the persons responsible for test automation overall in the respective teams.
Then we have the test automation engineers who are responsible for creating test scripts and to maintain them; sometimes they run them as well. And finally, we have the executors, the ones who are running the scripts, checking the details and, if something is not working fine, going back to the test-automation engineers and asking for support and help.

I rate Worksoft Certify at nine out of ten. I'm happy with the tool, I'm happy with our colleagues at Worksoft. We have a very good relationship, we can bring up everything. There isn't much I can complain about. I'm happy at the moment with Worksoft.

We have developed some end-to-end regression testing scenarios that we've found pretty valuable, so we have created a bunch of processes in Certify, linked them together, and we use them every week - sometimes more than once a week - in regression testing.

We used another tool for many years. It became unworkable because of the length of our scripts and how many of them there were, and how they were linked together. They became cumbersome in the other tool. It's much easier in Certify, and Certify can handle them, no problem.

In addition, our organization is implementing Agile, we're moving towards continuous development, and I don't see how we could do that, in any imaginable way, without Certify. We're able to import our changes weekly, based on the results in Certify. And we're confident that because of having tested the main business processes, fairly rapidly, within one day, we can tell whether the imports are going to break anything.

It has absolutely enabled us to automate and save time. The weekly imports of the changes allow the developers to plan on a weekly cycle, which increases the speed of their development. They don't have to wait for a release or anything else, they can test their changes quickly and get the results the next day. They know that they're able to import with no problem.

Finally, it has helped us cut test maintenance time. 

The idea that it's not language-specific is really nice. Keywords and the drag-and-drop functionality are great. The ease of use is superior to anything on the market. It's very easy to integrate. We've been very impressed with the tool. Because we primarily use the configuration with SAP, the integration is pretty seamless. But we have used our own in-house VB app as well, and it's worked very well with that. We've really not seen any problems whatsoever with integrating.

We love the Capture 2.0 feature. It seems to work very well. As for how long it takes to create documentation using it, we do not get into the documentation so much. That end is not as useful to us. But it's built-in if we ever needed it. We're not USDA or anything like that so we don't have a super need for documentation right now.

Also, the ability to work with the data, with recordsets, and plug those into the scripts is very easy and very powerful. We use it extensively.

In the past, when we've tried to automate some of our web apps, it has not been as robust. If there were one thing that could be improved, it's interaction with web applications. The issue we were running into is that it was harder to identify the objects than it is with some of the other architectured applications. That's the nature of the beast with the web as well.

It's very stable.

We have had some issues. We would be unable to log in, in certain situations. But they've all been self-inflicted, changes that we've made on our side that have prevented us from being able to use the tool at times. Once we got those resolved, we were fine.

It's fairly easy to scale, which is a nice thing. Once you create what I will call a sub-process, if you want to use that sub-process in many other processes, it's really easy to use. For us, that's what makes it scalable. You can use that same process wherever you need it. The use of the recordsets just allows us to be able to change the data that make it unique and that make it easily maintained. It's very easy to scale. It simplifies our workflow.

Worksoft technical support is very helpful, very knowledgeable. Whenever we've had an issue, they've responded very quickly. We don't actually have very many tickets, but whenever we've had them in the past, I've just gone into the portal and I get an email back, usually the very next day. I've never had to escalate an issue.

We were using Micro Focus UFT. It became completely unworkable for us. Our end-to-end processes were just too cumbersome for the tool to handle. It got worse and worse to the point where we had to say, "You know what? We have to change tools, this is not helping us." That's when we investigated Worksoft, and we were very pleased with how it worked.

The initial setup was pretty straightforward.

The last time we did it, for the upgrade into version 10, it just required me and one other person on the database side, and then the technical person from Worksoft. It was fairly easy. It took just a couple of hours. At that time, we were just upgrading. The basic architecture was already there so it didn't really require a project plan or anything like that. Once we got it set up, it was just a matter of migrating what we already had in UFT.

There have been several times where it has highlighted an important issue. Some of the defects we've found have been high-impact defects that would've really been costly had they made it to production. There are other times where, because we were able to test with Certify, we knew within a day whether there were gaps in the way we configured a change, things that we had missed that we wouldn't have been able to find if we didn't have the ability to test quickly.

That one defect we found easily saved us $1,000,000. That was just one. Over the years, the amount of money that it has saved us is certainly in that range.

The ability to test quickly has enabled us to develop quickly. We've been able to capture lots of savings in terms of projects that have been delivered faster because we can test faster.

There are savings on a lot of fronts because of this solution.

We would purchase more licenses right now if they were cheaper. Pricing is a little bit of a hindrance.

We didn't look into any alternatives.

It's a highly powerful tool. It's very customizable. It's not a cure-all for everything, but if you want to do end-to-end testing, regression testing, it's a great investment.

We use Certify for end-to-end testing of packaged applications. We have implemented almost anything that touches SAP, using Certify. When C4C came out, the customer application, we regression tested our existing suite to make sure that nothing would break. We anticipate doing the same thing with Success Factor. At the moment, we don't use Certify for web-UI testing, but we're planning on implementing some of that, coming up.

Since it has been up and running, we've had three people maintain it: Myself, I'm the principal QA person, and we have two offshore partners whom I've trained on Certify and they are now helping us execute and maintain the tests. It requires full-time maintenance. We have plans to expand the reach of our automated testing, so we plan on adding more people. We are the only three using Certify in our organization at the moment.

It tests our core business processes but we still have many core business processes that we would like to add to that, to validate if they work, before we send changes through every week. And we would also like to increase the speed at which we can add changes; not just once a week, but eventually daily. We plan on increasing our resources from a manpower standpoint and also from a technological standpoint. We're just going to try to do that as fast as we can. There are a lot of business processes that we would like to add, a lot of apps that we would like to add. The business side has continual, increased demand in terms of things that they are working on and they would like to automate and not test manually, so there's a lot of demand on us right now.

I would rate Certify at nine out of ten. I rely on it every day. It's a great tool, and any problems that we have are hardly ever attributable to the tool itself. It's always some other factor; the way we're using it, or some external factor, which is the problem. It's nice not to have to worry about the tool being the issue. We're very enthusiastic users.

It is used for codeless automation. I am using its most recent version, and it is on-premises, but we are using it through some virtual machines. It is installed on some virtual machines, and by accessing those virtual machines, we're using Worksoft Certify.

It is codeless. Typically, you have to write code in Java, Python, or another language to automate web applications or ERP applications, but Worksoft Certify is completely different. You do not need to do any coding, and you do not need to know any coding language for automation. It is completely a techno-functional tool. You can just enable the application that you want and start automating. 

There are certain tools in the market for writing the code, but they are not supported with some of the ERP solutions available, such as SAP, Oracle, etc. Worksoft Certify is not like that. It supports each and every ERP solution available in the market.

An organization doesn't need to hire new people for automation. For example, if you already have people who are well versed in SAP, you can just train them on this particular tool, and they'll be able to do automation for you. You do not need to hire any new resources for this particular tool, and even if you are hiring, you would need a less number of people. It saves costs in terms of resources. 

Scheduling and lights-out testing are very easy. You can schedule any number of test cases on a recurring or on-demand basis. So, you can run any number of test cases. Test maintenance is very easy, and you can very easily manage all your tests, which also saves cost. So, it saves costs in terms of resources and maintenance.

The Worksoft Capture feature is most valuable. For example, if you are creating a sales order in SAP, you do not need to go to each field and do everything. You do not need to write code for each and every line. You can just turn on the Worksoft Capture feature and manually perform your actions. It will capture all manual actions, and it will give you the steps. It will write the steps for you. 

They have a scheduler in Execution Manager, but it is not customizable. Its UI needs a lot of improvement. The lights-out testing is a bit difficult with that particular tool, and it needs a lot of improvement. Of course, there are so many integration options with Worksoft for execution, but when it comes to Execution Manager, which is their own tool, there is a lot of scope for improvement.

The integration with mobile needs to be improved. Initially, they used to support certain applications, and now, they are supporting all the web applications, but with minimal knowledge, it's very difficult for any tester to automate web applications. That's where they need to improve a lot. They are already working on it. They have given additional features, and with the help of those features, you can easily automate, but they need to keep making it easier for business users who do not want to get into each and every technical aspect of it. They just want to capture the actions. It is working fine with SAP, but they have to concentrate on web applications. They also need to support Safari, as well as macOS, better.

If you keep on running it for a long time without removing unnecessary things, the load on the database increases, which impacts the performance of the tool. Sometimes, it hangs or is slow. We have faced this situation with the earlier versions, such as version 9, but in the current version, there is an improvement. The slowness has drastically reduced, but a lot of improvement is still required in this particular area.

Another negative of this tool is that its license is costly. It is a bit more expensive than other tools.

I have been using this solution for close to six years.

Its stability is very good. It is reliable.

Its scalability is very good. We are a very big organization. In my team, we have close to 14 automation engineers, but there are so many other projects where Worksoft is used. I don't know the exact number, but it would be very high. Its usage is not extensive because it is mostly being used for ERP applications. 

In my previous organization also, it was implemented for many projects.

Their support is excellent. When you raise a ticket, within a day, you get the solution. If they're not able to resolve the issue at their end, and it is on our side, they connect with us and show us what exactly is happening. They check everything. 

Its initial setup is very simple.

It was implemented in-house. We get some files that we can directly install, and that's it. We do not need to do anything other than double-clicking the given files, and it creates everything automatically.

It is costly. It is a bit more expensive than Tricentis Tosca and other tools. If they reduce its price going forward and have partnerships with other vendors, it will grow like anything.

Tricentis Tosca is a strong competitor. Worksoft Certify is more expensive than Tricentis Tosca. The solution that we use depends on a client's choice. 

In the long run, most people will prefer non-coding tools, where 0% coding is required. It is, for sure, a tool that would be useful, but there is only one hiccup. If you are executing tests multiple times, you'll have various results, such as pass, fail, etc. When you are running it for so long time, the file size on the database will keep on increasing. So, you have to keep on doing the maintenance. For example, if you need the results for 2018, you should download everything. You would need some kind of storage.

In many organizations, the managers think that if they hire one or two resources with Worksoft knowledge, they can train everyone in their team, and build a good team, but that's not going to happen in the short run. That's why I advise having two to three dedicated people to train other people and build a team. With only functional team members, it is very difficult to build a proper automation team. I have seen an automation team that wanted to train 20 people. Currently, we are training 100 people. There are three of us who are training 100 people, which doesn't work out very well. So, there should be a few dedicated people for training and implementation. 

I would rate Worksoft Certify an eight out of ten. It is a fantastic tool. Its maintenance is easy. Building test cases is easy, and planning is easy. Almost everything is easy, but the look-and-feel of the Execution Manager scheduler is not good, and currently, customization is not available, and product creation is not available.