What needs improvement with Worksoft Certify?

The product could be improved in terms of web applications. One challenge I faced was that web application parameters often change in the back end, affecting the automation scripts. Small changes in the HTML page design can impact the automation process, unlike SAP, where the script remains stable unless there is a functional change. Therefore, maintaining automation for web applications requires more effort. If my script is long and involves a large data set with over a hundred entries, it can slow down or become inefficient. I suggest improving how the platform handles large datasets during lengthy scripts.

Some features are missing from a testing perspective. You need to know how to connect everything to create requirements and stability metrics for the routine.

The technical support of the product is an area of concern where certain improvements are required.

The primary area for improvement is the support service. The support process can be time-consuming, with tickets sometimes remaining unresolved even after considerable follow-up. Also, it doesn't offer a training or trial version for users to explore the tool before committing. Enhancing direct connectivity between radio and Worksoft Certify would streamline and improve efficiency. Although there are YouTube videos for specific features, the Worksoft community lacks corresponding content. This has been a common concern among users who find information on YouTube but struggle to locate it on their community portal.

The solution's support could be improved. Worksoft Certify's support team should respond more promptly when we are stuck with certain issues and looking for a solution. They have been helpful so far, but quick conversations can happen if they function within the timelines. Worksoft Certify has already come up with a new tool, a continuous testing manager for lights-out testing. It's good that they are working in those areas to develop new tools to support the lights-out testing. Worksoft Certify should also try to develop some solutions that can be more beneficial for non-SAP applications, like web-based automation or Excel-based automation. Worksoft Certify can be made more robust in those areas.

We can't get the process intelligence module to work properly. We can't get the impact comment that analyzes the incoming development code to run, either. We've also had bugs in the CTM and execution manager in the past year. It took technical support a long time to resolve this issue. We escalated it so that the vice president of the company was included as well.

The problem with Worksoft Certify is that it's not always stable. It runs on a live platform that's constantly changing, so the test script needs to be adjusted every time, which is very painful. That's the pain point in Worksoft Certify, and if Worksoft could make the solution more stable and more "change-proof," that would help my company greatly. A feature I want to see in Worksoft Certify is for it to be able to work on test cases on mobile devices, though that could be difficult. I also wish to have more portability in the solution in terms of the script because, in the SAP environment, my company has to run Worksoft Certify in more than one system, so if Worksoft can make the process more portable, than would be fantastic.

Worksoft Certify's tech support's response time could be improved.

I would like the Worksoft web application to be more robust. For example, SRM has different modules in SAP. When we automate some of those objects in the web app, it's initially fine, but after a couple of executions, we need to again record that particular object or get Worksoft to identify that particular object again. It works well with the SAP R/3 or GUI but there are limitations with the web application and Worksoft needs to improve that area. In terms of additional features, we'd like to see the default report which we generally get after the execution is completed. There should be an option to customize reports according to our requirements. Tosca provides that feature but it's unavailable in Worksoft.

What could be improved in Worksoft Certify is its integration with other tools, for example, test management tools such as Jira, ALM, or any other test management tools. That integration is missing.

The overall speed and performance of this solution could be improved. In a future release, it would be useful to be able to do API testing.

Worksoft Certify needs improvement on customization of reporting and how you report final outcomes. It needs more customization.

They have a scheduler in Execution Manager, but it is not customizable. Its UI needs a lot of improvement. The lights-out testing is a bit difficult with that particular tool, and it needs a lot of improvement. Of course, there are so many integration options with Worksoft for execution, but when it comes to Execution Manager, which is their own tool, there is a lot of scope for improvement. The integration with mobile needs to be improved. Initially, they used to support certain applications, and now, they are supporting all the web applications, but with minimal knowledge, it's very difficult for any tester to automate web applications. That's where they need to improve a lot. They are already working on it. They have given additional features, and with the help of those features, you can easily automate, but they need to keep making it easier for business users who do not want to get into each and every technical aspect of it. They just want to capture the actions. It is working fine with SAP, but they have to concentrate on web applications. They also need to support Safari, as well as macOS, better. If you keep on running it for a long time without removing unnecessary things, the load on the database increases, which impacts the performance of the tool. Sometimes, it hangs or is slow. We have faced this situation with the earlier versions, such as version 9, but in the current version, there is an improvement. The slowness has drastically reduced, but a lot of improvement is still required in this particular area. Another negative of this tool is that its license is costly. It is a bit more expensive than other tools.

We can use it for the web application, but we are facing stability-related challenges. The properties are getting changed. For example, when I am performing any operation on the text box but the development team has done some changes, our Worksoft scripts are getting changed. This is the main challenge that we are facing while developing tests for the web application in Worksoft Certify, where any changes in the backend are indirectly impacting our scripts. For the web application, there is a scarcity of resources. Unlike an SAP application that doesn't require much experience, for the web application, you require experienced people. We also use XF definitions for the web application, but we can't create them on our own. So, we need to take support from the Worksoft team, and we have a dependency on them.

There are some other more complete tools than Worksoft Certify, such as Tricentis Tosca. It has a quicker way of taking in a customer's feedback with more efficiency. I do not see Worksoft Certify having a lot of progress over the years that we have used the tool in this area. From 2014 to 2022, there were fewer improvements in the UI looks and feel. Overall there were fewer improvements in the solution. There were some challenges that we faced with our customers, and Worksoft Certify provides customer-specific call announcements. For example, customer A is trying to work on Worksoft Certify, they will face a specific challenge in utilizing the tool, and Worksoft Certify provides a custom solution to them. With customer B, they are faced with other challenges, and Worksoft Certify provided them with a customized solution. There is a chance that the same issue that was faced with customer A, customer B had also. Instead of Worksoft Certify providing a dedicated solution for everyone, they were providing solutions on a customized level. They can improve on this process. There were some challenges that we faced with respect to automation. For example, there were some areas where we had to do a drag and drop of some of the objects from one place to another. In some of the areas in SAP where you have to perform a drag and drop, that feature was not available in Worksoft Certify. We had to find some alternate ways of doing those things.

Initially, there were challenges because there is a concept called XF definitions, where each application type, e.g., the vendor and workflow, has to provide these XF definitions. Last year, these XF definitions were not provided. However, whenever we face any issues, we have to raise a support case, then they update the XF definitions in our enrollment. This year, they improved that and have the last XF definitions for SAP Fiori updated as of February 2021. For integration, projects are usually agile. The customers are looking for integration with CI/CD tools, like Jenkins, Jira, Xray, Zephyr, etc. There is no clear documentation on how to integrate Certify with these tools. Also, we didn't receive the required support when needed. Worksoft used to have webinars on this, but those webinars used to be on after the integration was established and how the integration works, not on how to create an integration. They should come up with a solution on how to do the integration. Jenkins, Jira, and Certify should be the same in every company. The only thing required is the pipeline code required to integrate Jenkins, Jira, and Certify if they make it available for everyone. Also, if an expert team could help customers to integrate, then that would really help our customers a lot. As part of our weekly regression, we wanted to use Execution Manager. However, from 2017 until March 2021, Execution Manager was not working as expected in our enrollment. It could have been better. If Execution Manager had worked well, then we could have doubled our productivity. Unfortunately, it had problems.

Certify is integrated with Solution Manager, but this integration could be easier. Overall, in terms of how it is working, I find it pretty clever in all the areas. There are only tiny things. For example, to log into Certify, you have to put in your username and password. In version 12, they changed it, and the password is no longer stored. So, you have to enter it every time you log in. Similarly, there should be a way to store the layout of tables in Certify. You can adjust your tables, but when you close Certify, if I recall correctly, the layout of the table is not stored automatically. So, you have to adjust it every time. I'm, however, not quite certain about it. These are tiny things that they can improve, but compared to the whole feature list of Certify, they are not so important.

We would like this to be able to be used outside of SAP applications, as it would be good for other types of products.

When you are using an older version of Worksoft Certify, like version 10, Capture will be running under your status bar. Then, you don't know what transactions you are currently recording. In the old version, you don't have the visual of the capture. It works well with other tools, but there is some integration required with Solution Manager. If you are using test repository in your solution manager, then there is no direct integration between Solution Manager and Worksoft Certify. There is an improvement needed in the reporting within Worksoft Certify. We have either a detailed report or a summary report. We don't have report that can be used for training purposes. A different tool from Worksoft has to be utilized for this. Worksoft Certify needs a bit of improvement for its web-based processes. It can be difficult because you need to recall the maps, then you still have to add-on for your browser. When you are using the browser-based testing, you cannot even move your mouse or do anything on your system when you are using the web-based testing. Therefore, it needs a bit of improvement on that side. While it does work, it needs improvement. From the SAP side, there is nothing better than Worksoft Certify. However, from the web-based, we are moving towards Fiori. SAP will soon be totally web-based. For Fiori, they need to be great with SAP testing. Thus, Worksoft has to improve the web-based testing part for Certify. They have a lot of versions coming out every year, like four or five versions. They need to reduce this number. There should be one or two versions every year with add-ons, if necessary. Because if you want to upgrade your Worksoft Certify from an older version to a newer version, you almost need to reset the solution and we don't have that much time. We cannot dedicate four times a year to having the newest version of Worksoft Certify. Though, if I don't need the need changes or improvements, I can skip the most recent upgrade until the next version comes out.

One caveat is that if you start running models in different parts of the end-to-end process — when you really try to hit the sky and make everything automatic, to cover multiple supply chain tiers processes in one e2e test, or similar processes that are really complicated — then tool simplicity turns into disadvantage. On other hand it stops us from unnecessary over engineering the test automation framework. Architecturally, because Worksoft specifically built a database-oriented application, you essentially store code in the database. Git and text files orientation is more traditional approach with boundless set of tools to control versions, manipulation and analysis. But at the same time, Worksoft supplies us with their own version control inside Certify that has sufficient functionality for now. When it comes to mobile testing, we have a small bottleneck there. You have to buy third-party separate licenses if you want to test on a mobile. Business wise we see room for improvement there, although it's that really critical for us. None of these are showstoppers for our operations. Worksoft proved to delivery significant improvements in last 3 years and more we wait from 11.5 version. Overall, we are quite well covered with test automation related tools and nothing special is needed.

I would like the ability to more easily modify the report from the Capture feature. One of the things I don't like is that it keeps repeating all the field selections throughout. To me, if we put them up front, we shouldn't have to repeat them at the different steps. It should just be Pass/Fail and show the screenshot. I've talked to them about this in the past. There's another part of the Worksoft suite that probably does a better job at documentation for training purposes and providing an understanding of business process. It's the Certify BPP which we're not using right now because we're really focusing on automating all these different ERP systems. Whereas the testing is very detailed, which is great for the auditors and it's great for the users because they see everything we're doing, it makes for some big PDFs. It's a double-edged sword. Also, with the codeless process automation across packaged applications, once in a while, if we get a weird application that's not widely used, it gets a little stickier. First, the software has to learn the fields, so you have to identify all the fields. Once you do that, as long as there isn’t any non-standard code in the application, then it works fine. But there's that one step that you have to do, a step you don't have to do with SAP and Salesforce, for example. In addition, Worksoft definitely needs to continue the march toward bringing in more and more of the software that people commonly use. They're doing that, but they can only march so fast. I know Worksoft is doing some stuff with RPA. There are other tools that strictly do RPA, but aren't automated testing so I'm not sure if they will be able to compete with those. I know that we did do some automation, what we call "bots," with Worksoft, and it was clunkier than some of the RPA tools that are currently on the market. I suspect that they'll come up with a very competitive offering. I would also like to see some better reporting of testing status, reporting that we can easily generate to say "Okay, we're 50 percent done and we've got 10 fails and 800 passes." That's what test management software is for and Certify integrates with that. Bang-for-buck, it's probably not a great place for Worksoft to invest. They're probably better off with RPA and bringing on the ability to more easily test software, like Salesforce and CPQ. I'd love to be able to do that as easily as I can with SAP. I would like that same ability to use Capture in CPQ, instead of using Silverlight.

If we can import process from other automation tools like UiPath, Selenium, or Automation Anywhere, then it would be more helpful.

Pricing is a bit high and we would like to have the availability of a trail environment for beginners and training would be great to have and easier to expand and use by more and more consultants. The bigger reports needs to be more customized by the tool for better use, also tool also can be further simplified by a better Graphical user interface (GUI) will help us a great deal while taking up projects. We look forward for these upgrades so we can enjoy using the tool more and help others learn this tool.

Looking at it as a product fully packaged, I would like to see more documentation or ease of use of the documentation. Sometimes documentation does exist but we have to search three different sites to find the proper way to do things or track down the technical document that explains certain fields. That, in turn, relates to the ease of use and how objects interact with each other. The application could lend itself to be simpler. Another area that I would like to see improved is how the permissions are applied. If you're applying permissions groups to a user, one of the options is to delete the group entirely and lose the entire permission group, rather than just deleting the permission from the user, which seems a little silly. In my opinion, that whole module of permissions is very confusing and lends itself to common errors. We have to rebuild permissions occasionally. The functionality is all there. I just think the way it's packaged can be confusing. We are successful and we can get things working the way they're intended to in Worksoft. It's just that sometimes finding how to do that, or where it is described, can be difficult.

Performance on the web UI part, especially with some of the more comprehensive Fiori features, like the complex tables that are being used, could be improved. In those cases we have noticed a lot of execution-time increase with regards to the Certify solution.

For Execution Manager, I would like it to be more robust interface and be able to view the remote machines full screen instead of a little window. This would be a great upgrade for us. I would also like more customized reports without having to print out big reports.

For business users, the product needs to be more intuitive and user-friendly. The updates for SAP Fiori have been great, where previously we saw a lot of issues. A year ago, it used to fail miserably. Worksoft teams worked with our product teams to resolve this issue. I would like to see more integration features. It needs to provide the automation across the right set of plugins and integrations, along with cloud solutions, with the ability to quickly adapt. Going forward, I am looking for end-to-end testing in DevOps.

I would like more reporting in analytics. There is a lot of manual work for us as program managers and test managers which has to do with supporting our value statements. E.g., if there is some way that we could systematically capture how long it is taking for automation processes to execute, then we could insert some notes as far as here is how long it took for them to do the manual capture. Then, we could calculate time saved and have a formula for savings. If they have some templates that we could all partner, there are a lot of customers who have created their own, but with the new companies coming onboard. Do they create them on their own or do they try to leverage the best practice within our customer community? There is more we can do here.

There is a learn functionality where Worksoft learns applications that would be nice if Worksoft expanded its support for other applications that aren't web-oriented.

We are not using Certify in the development area, only in the functional or end-to-end areas, and there is a lot of activity going on in the development area recently. Right now, the development teams are using open source tools, like Jenkins. This would be a game changer if Worksoft could start in the development area. Going forward, Worksoft will be integrating with Jenkins, which will be great for us. A part of our CI/CD pipeline, we have to deploy through Cloud AWS. So, it is good to hear that they are moving to AWS too.

We have had run ins with some bugs on Business Process Procedure (BPP) and Execution Manager. We also did upgrade Worksoft Certify recently to clear up some issues with server fogging.

First, product stability needs major improvements. Our projects get delayed and our executions fail because the product is not stable enough. We hear consistent reports of performance issues, Execution Manager crashes, and so on. What is not helping is Worksoft's ability to respond to issues. They do not have clear SLA's on when issues get closed. We have had High criticality issues that were open for months. Second, there are mixed results for non-SAP Automation. We tried to do web UI testing on SAP Fiori and some internal applications, but the results were mixed. In some cases, we are able to automate, but it takes a very long time to do it. There are other cases where we totally couldn't do it or our customers back out somehow, because of the length of time or limitations of the technology. Also, Worksoft testability assessments take quite awhile.

We're really hopeful for the mobile testing in Worksoft Certify going forward.

I am looking forward to a feature which will be available in the next release, version 11. The search feature, where you can search for any process that you have created. I would like BPP to have more filtering options during the report creation. This would make our customers happy.

It is very easy to use, but there are some places where they need to improve their security. E.g., the BPP tool is just a URL, which does not ask you for a username and password. Anyone can login and can see it. This is a challenge. Regarding the Execution Manager, which is good, since it comes with a username, password, etc. However, once you are logged inside, anyone can execute anything. It shouldn't allow this. Worksoft should create restrictions at the folder level structures, or somewhere they know they can align only few people to do the job, instead of allowing anyone do anything that they like.

We would like it to support mobile automation going forward.

I would love to see more data mining modules. I want to see more stability in the Execution Manager.

I would expect more opportunities to automate Java. I would like it to analyze what we are not using. I would like to see the impact analysis integrated with the performance testing tool. We have multiple tools doing multiple items. I would like to have one common tool. I'm also interested in load testing automation and whether we can create a script for it, then can we use the same script for my performance testing?

We are looking for some enhancements on the Capture 2.0 tool. This would give us the ability to control it directly, like we could with Capture 1.0. Right now, Capture 2.0 doesn't really work for our Business Analysts. Because they don't work with it all the time, it's a little complicated for them to stay up to speed on it. With Capture 1.0, we wrote a wrapper to make it easier for them to use, but we can't use that wrapper with Capture 2.0. So, if Capture 2.0 gets enhanced, we'll start using it. For an enhancement, we want to be able to start and stop recording through an API. Then, we want to see how many steps have been recorded through the API. We do a lot of test maintenance because they are constantly changing the applications. This is one of our biggest problems that are constantly making changes and switching products. For example, we used to use the Supply Relationship Manager. Now, all those tests that we build there will be replaced when we go to Ariba. All the old SAP GUI stuff for HCM, when we got SuccessFactors was thrown out, then had to be redone with SuccessFactors.

A feature that I am looking forward to in version 11 is a search capability, where you can search within the script themselves for keywords. That will be really helpful.

We are interesting to do better, year-by-year. At the moment, we are doing automated regression tests. The next step would be DevOps or artificial intelligence. Our programs should also develop in this way. We want to have automation everywhere where it is possible. Therefore, we need more options for these next steps. We have used Capture, and it works with Worksoft Analyze. We had some experience last year in August because we started our schedule 1.0 and used Analyze in our tests. We were using the central site for one year with our ERP testing and were quite successful, but this was with Capture 1.0. Then, there was a change to Capture 2.0. In the end, there have been some challenges with the newer version. Therefore, the company testers, the local ones, do not want to use Capture 2.0. 40 percent of the test cases were finalized with automated capture and automated documentation, then the others were done manually. Because we have to create test nodes, we were asked to create a tool that automates documentation, which was Worksoft Analyze. However, with the switch to Capture 2.0, we had some challenges in the beginning. What we did afterwards, together with Worksoft, was we sorted through all the known bugs. So, at the moment, we don't have any known bugs open. We will retry this year in our central test first to find out if it Work Analyze is fine, then if it is okay, we will continue with the local test teams, as well. On the Capture 2.0 topic, we were not satisfied, because we had a version that wasn't really tested from my perspective. Of course, Worksoft said it was tested, but we found a lot of bugs. In the end, our national company and local test teams did not use Worksoft Analyze because it stopped working. We have ten steps, then on the ninth step, it stopped working and we would have to do it again. So, they stopped using Worksoft Analyze and Capture 2.0. However, this is solved. Worksoft directly helped us to find out what the bugs are, and solved them. Then, we retested it. At the moment, we don't have any open bugs. On average, it takes one day for Capture 2.0 users to create document. We expect it is faster, but you have to do it several times sometimes. You have to check the documentation that everything is fine.

One feature that could be added to Capture 2.0 is generating a PDF file from your capture, so you can see your screenshots and steps. This will really help teams leverage the documents generated as part of requirement/training. Right now, when we do regression testing, we manually have to generate all the reports and populate all the results in HP ALM. We really are looking for a solution to have send all the results to HP ALM once Execution Manager completes the execution, then automatically logs them. Our offshore teams experience a lag/delay when using the Worksoft interface. As of now we use VPN and Remote Desktop to help us with this issues, it was be great to see how much Certify 11 has improved in terms to offshore accessibility. Every time there is a new release of Worksoft, they present it in a conference. However, there is no training document nor one point of solution where I one see what new changes/feature have been implemented, like a portal. If I don't know how to use a feature, there is no training nor documentation available. When you reach out for support, it takes time for them to research it and get back to us. I would like more use cases or at least a weekly email update to all the customers saying, "These are new features which have been included in the last week." That would really help.

We had a lot of issues with the optics changing because they're dynamic. We just recently learned they are already worked on fixing it. Another feature that they are also working on is being able to export processes from one project and upload it to another project. Therefore, we can change our both structures within Worksoft. We prefer to run and set it up based on business use for separate projects. It's exciting to know that this is coming. We want to be able to move assets from project to another project without having to be in the same project or the same folder structure. I would like the GUI to be more user-friendly and intuitive. E.g., previously, the buttons were gray. Now, they have color to them and are fun for the user. It also makes them easier to identify.

I would like to learn how to get better logs for their support team.

The product had some UI issues. In the next release I heard the UI issues will be lifted up (version 11), I am excited about it because the product will have more UI features. We are thinking of upgrading our existing Worksoft Certify from 9 to 11, when it is released, as this will be good for the company and help all our users.

I would like Worksoft Certify to do automation at any layer (the UI layer, API layer, or database layer) and challenge competitors in the RPA industry, like UiPath and Automation Anywhere. A lot of customers are looking at testing, not just at the UI level, but testing the application or their ecosystem at the API layer. Worksoft could invest on testing on APIs. There are some open source tools available in market which do this, like SoapUI.

We went into this with the thought that we wanted to be able to hand this off to a business user, so the business user could develop their own test cases automatically through automation. We are not seeing that. We still have it assigned to an IT professional, someone who is certified in Certify. We constantly have to have that type of person around who can build these test cases for us. At the moment, there is not an automated testing tool out there that will allow a business user to develop their own test cases, and certainly not at the level that we want it to be it. So, this may not have been a realistic goal on our side to expect that one of our business people, who has their real job, could spend a couple hours here and there developing test cases on an automated testing tool, like Worksoft or any other. It's a software package, and you have to know the software to be good at it. You have to have a certification in the tool to be able to be really good at it. It would be great if our business testers could develop their own automated test cases. However, we either have to bring them up to a level of certification on the software or go hire somebody to do it. Worksoft, in essence, is the Mercedes-Benz of testing tools. If you want a Mercedes-Benz, you have to pay a bit more money. With every release you do, you have to go back and touch your old test cases and bring them up to speed, or develop new test cases. In the beginning, that is a challenge because you have to have someone who is certified in the tool to help you develop these test cases. It is a little complex for someone who is not in the autotesting space to learn it. Like any software, you don't show up to use Oracle Database on day one and think you know it. You have to learn it, get certified in it, and understand it. This tool is similar in that sense. You have to have someone who knows the tool and knows how to use it. It's not something that your business users are gonna pick up, especially if they have a day job. It will take a long time for them to pick it up without full dedication and going to get certified.

One feature that we have been asking for has been to treat tests as code and store the source code for tests in a configuration management tool. Right now, for version control of testing, it's all internally within the tool. If we have a test of a business process and want to revive that test, our methodology now is purely manual work. We go into the tool, create a copy of the existing test, and call the next one: v2. Now, we have two of them and the only way you can tell them apart is by its naming convention. This is not an efficient way in terms of how modern applications do version control. If this was code, we could plug it into a tool like Git or GitHub to manage of our versioning and branching. The reason why we want to do this is that the application which we are testing branches. When we branch the code, we put a bunch of new functionality on the new version while our production version stays unchanged. Then, at the end, we merge the two together. From an automation testing perspective, we have to run tests on both. Then, we have two current versions of our test. So, it's a bit hard to manage in the tool right now because you can only have this manual approach where we are tracking it via the name convention. Whereas, a modern way of doing it would be to have our application plug it into a version management tool, like GitHub, where we would store the code and could just pull in the version of the test that was applicable to the version of the software that we were testing. This is something we have been asking for for a while now. I understand that it's in the pipeline, and it may be in their latest version (version 11). This is something that we will be looking into this quarter. The challenge that we face everyday for test automation are more internal (people issues). We need change management and getting people to accept automation instead of the technical limitations of the tool. The tool does what we need it to do from an SAP testing perspective.

We struggle sometimes with the web side of things, although, I don't know if it is the product or the web application. When you get to the web, Worksoft needs to learn all the fields, and sometimes, it doesn't recognize them. We have had a few cases where we reached out to Worksoft, and said, "Can you help us through this?" because we were not able to do something. Sometimes we have to think about using a different tool. However, the problem with some of these other tools is that they usually require a certain level of data programming skills, and with Worksoft, you don't need that. Almost anyone can learn it if you know how to use the application that you're trying to test. So, this has been a little frustrating for us. We were getting a little frustrated with a tool called Worksoft Analyze. What the tool does is it will look at a transports that you are about to move into production, and review them against your production environment to identify all things that you should be testing by identifying all of the objects thee transports are touching. I would like to use Worksoft in a cloud application rather than the RDT set up we have currently. We are starting to move to S/4HANA, away from SAP ECC, and into the cloud with S/4HANA. I'm doing an S/4 Hana implementation project currently. If we could access the S/4 HANA quality system Worksoft Certify application in the cloud, it would solve most of my infrastructure issues. I would like an improvement in Worksoft’s web application interfaces. I know Worksoft has been focusing on certain frequently used applications, such as Salesforce. They are doing what they did with SAP to make it seamless and put all the field definitions in future upgrades. They should continue down this path. It would be nice to have a way to learn information instead of field by field. I would love to be able to have a more customizable test results report. Typically, if you run 500 automated tests and two fail, you send those results to the Functional Team for remediation. However, we recently did a big service pack update in SAP last year, where we went up six or seven service packs in one shot. Thousands of transports went in and we literally had to test all functionality. All of it. So, we ran hundreds of scripts. That team wanted to see the results of every single test. A customizable report would have decreased the file size we sent out for all of these tests.

We have requested for some minor new features which Worksoft is considering. The PRIMO image recognition functionality has room for improvement, especially around its ability to work with java interfaces, Execution manager scheduling, etc. as we have observed. As we explore more of our legacy systems, I am certain there will be a need to use more of the PRIMO features to learn the objects. Overall from a SAP perspective, it works almost seamlessly.

There are a couple of small things, technically, that could be improved. Features we have asked for include single sign-on. It's a bigger project to make sure that our end users do not have to store passwords, usernames, and the like, for the different tools we have. We are also working on an additional integration with another tool that we have in place for lights-out testing. That's ongoing at the moment. Another idea we brought is that the definitions for the objects need to be automated. They need to be recognized automatically by Worksoft Certify instead of changing them back and forth manually. This is also something that Worksoft is currently working on. Updates, in general, is a topic that we are working on with Worksoft on a regular basis. For new products, for new UI technologies when they come out, the test-automation providers need to update their definitions to make sure that the objects are recognized correctly.

In the past, when we've tried to automate some of our web apps, it has not been as robust. If there were one thing that could be improved, it's interaction with web applications. The issue we were running into is that it was harder to identify the objects than it is with some of the other architectured applications. That's the nature of the beast with the web as well.

It is poor for a web based application. We are living in very integrated organization are most of the companies out there. There's the big companies we have these organizations that we work with, where we have plenty of different services which are very much connected. So, we are really looking solution which really can support all the different services. We really need to focus end to end instead of stand alone case. Whatever reportings are there, so they're really not very user-friendly. So there is a lot of technical data instead of user-friendly data. It would be wonderful to add web support, I would really like to see this in the future.

Worksoft Certify can better identify web test objects by providing libraries through its Extensibility framework. It has improved with it's latest version through Extensibility Tools support and additional test object attributes. Object identification is a necessary barrier for the technical execution of a test script, aside from the script's needs to verify business rules. Automation engineers must affirmatively answer the question: Is the application automatable? If Worksoft Certify or any other test automation tool can get object identification down consistently and flexibly so that maintenance of web object identification (and hence the web test script) is easy and done with no execution problems, it will allow more scripts to be developed and applied faster with the human test automation staff resources on hand. A good part of the technical test automation work is to make the script run and affect script changes when the application under test changes. When done right, the business part of automating the test application functions remains the same - leveraging your test investment. Certify may not the best solution for your needs, but among the commercial and open source offerings in the automation tools space, it should rank very high. It is also a leader in rapid SAP test automation, and can be used as general all-purpose test automation tool for varied clients like PC, SAP, and especially Web. Only one or two tools I have experienced could be suitable in place of Worksoft Certify. However those tools also carry their own set of disadvantages.