It is slow compared to physical device testing. The interactive speed could be improved. And sometimes we have issues with our app not working properly with Perfecto. Even though it is working perfectly on the physical device, in Perfecto we find issues. That is another thing we'd like to see improved. Perfecto support said not to use the instrument option so that you do not face that issue. But unless we use the instrument option we cannot do the inspection. We haven't had this issue with all versions of our apps, only with a few.
Software Quality and Automation Specialist at PRIME CONTROL
Real User
2022-10-07T21:31:00Z
Oct 7, 2022
Sometimes, when the automated tests sync up or we have to debug remotely, we cannot interact directly with the device. We can interact with the code in the debugging tool, but we cannot directly click on the element on the screen or send other kinds of inputs to the device. This is possible with other tools. Also, it would be nice if there were some kind of API to get a list of available devices. Currently, we have to look at the web interface to see the available devices, but the pipelines can't do it on their own there. We always need to do this manually, so it would be better if this feature was automated.
Manager at a insurance company with 1,001-5,000 employees
Real User
2022-08-13T00:59:00Z
Aug 13, 2022
Going by the dashboard or analytics capabilities that Perfecto or Perforce is looking to offer in its roadmap, it will certainly help if they also cater to executing and enabling decision-making, rather than just focusing on standard testing metrics such as execution, efficiency, and defect rate. These are good metrics, but they don't necessarily enable decision-making for SLTs. Any improvements in the dashboards and reporting tools should focus on metrics or SLAs that can help with decision-making.
There was a discussion about having the capability to export the test results to a certain tool that we use in our project. If that were added it would be great not having to manually take screenshots, put them in a document, and share them on the different test management tools.
Director, QE Automation & Analytics at a financial services firm with 10,001+ employees
Real User
2022-07-31T09:45:00Z
Jul 31, 2022
I would like to see the inclusion of machine learning features. If we can have that, it will be a better tool. On the analytics side, Perfecto is in need of improvement. The product has a cool dashboard, no doubt about that. But that dashboard is limited to visualization and slicing and dicing the numbers, as well as getting the reports in multiple formats. What I would like, beyond that, is if we can have some predictive intelligence capabilities. This includes machine learning and modeling capabilities. I would really like to see that from Perfecto. As an example, I would like to be able to predict our failure rate in an upcoming release of our software. The reliability of devices is very good, at perhaps 99.9%. However, sometimes, we find that some devices are offline and not available. I would like them to achieve even closer to 100% reliability. I know that 100% is not achievable but they can still push that number. The timing of the software is an area that can be improved. For example, there is a new Appium release and Perfecto should be able to get it to us more quickly. They take their own time for internal testing before they roll it out to customers but we would like to use it faster, so it would be helpful if they can increase the speed of that process. It is somewhat related to improving their time to market.
Systems Engineer at a financial services firm with 10,001+ employees
Real User
2022-06-08T13:55:00Z
Jun 8, 2022
I can't really say anything negative about Perfect, but if we talk about feature enhancement, there are a couple of things that I'm working on with our customer support manager. I'm hoping they can support on-premises instances. We have been working on a JIRA integration with Perfecto—and I'm extremely impressed that they have that—but at this time they're not supporting onsite JIRA instances, which is what we have. I love the idea that the testers will be able to run either their manual or their automation scripts, and that the test report will automatically go into that team's JIRA instance. That will save some time. The testers won't have to extract the report, save it, and bring it over to their JIRA instances. Then, when the scrum master reviews the test results, if there are questions, they have all of the details from the report right there. One other enhancement that I've already spoken with them about is that they don't support Android for ADA (Americans with Disabilities Act) testing. They mentioned that they're going to try to begin supporting that later this year.
IT Quality Control at a insurance company with 10,001+ employees
Real User
2021-08-27T20:55:58Z
Aug 27, 2021
Its performance should be improved. Anything to speed up the user interface would be a great help. We've had a lot of pain with their migration from a product UI that was based on Adobe Flash to the new product that is based on HTML5. Migrations like that seemed to be very painful and not a real smooth process. We're still sort of recovering from that migration from old technology to new, and we haven't got all the functionality ported over that we used to have on the old UI. It is sort of unfair to ask, but we don't have SSO right now. So, we're working towards that. If that had been available from day one, it would have been super awesome. Once we get to that point, it would be a whole lot better. There could be some additional administrative tools and reporting tools to get more direct hands-on reporting for usage, user database, and things like that.
Prin IT Bus Systems Analyst at a healthcare company with 10,001+ employees
Real User
2021-02-10T16:44:41Z
Feb 10, 2021
We bought this solution because it is supposed to be proficient at both web and mobile testing. It does well for mobile testing, but when it comes to the web aspect, it is lagging a little bit in terms of execution. Specifically, if it is a native app then it does really well. But, with a hybrid, or it is a native mobile web app, it seems to be lagging.
Regional Head Customer Experience at a financial services firm with 10,001+ employees
Real User
2020-04-13T06:27:00Z
Apr 13, 2020
We have had challenges while using this product. The main problem has to do with OTP. OTP comes on the SIM but there is no SIM card in the cloud, so it makes it a big challenge for us. It is available in development but because we don't have the OTP API exposed, we don't have it in production. We have had some issues with performance, which is something that should be improved. The price of Perfecto Mobile should be lowered.
Responsible for Test Management at a tech services company with 10,001+ employees
Real User
2019-11-18T07:22:00Z
Nov 18, 2019
I looked at the solution a year ago. I'm not sure what functionalities were lacking or need improvement, but I do find it to be a good tool. It would be ideal if there was a complete integration with other test management tools and other applications like HPLM, Micro Focus, or Microsoft Azure.
Perfecto is a cloud-based mobile app testing tool that is designed to help test applications on different operating systems and devices. The solution lets users automate processes related to advanced test scenarios and access browser versions and devices according to individual requirements. With Perfecto’s smart test execution capabilities, teams can manage testing across web and mobile platforms using a single script, conduct parallel testing for large test suites, and design advanced...
It is slow compared to physical device testing. The interactive speed could be improved. And sometimes we have issues with our app not working properly with Perfecto. Even though it is working perfectly on the physical device, in Perfecto we find issues. That is another thing we'd like to see improved. Perfecto support said not to use the instrument option so that you do not face that issue. But unless we use the instrument option we cannot do the inspection. We haven't had this issue with all versions of our apps, only with a few.
Sometimes, when the automated tests sync up or we have to debug remotely, we cannot interact directly with the device. We can interact with the code in the debugging tool, but we cannot directly click on the element on the screen or send other kinds of inputs to the device. This is possible with other tools. Also, it would be nice if there were some kind of API to get a list of available devices. Currently, we have to look at the web interface to see the available devices, but the pipelines can't do it on their own there. We always need to do this manually, so it would be better if this feature was automated.
The flakiness, or the accuracy, of the test execution can be improved. Also, the responsiveness of their cloud lab could be improved as well.
Going by the dashboard or analytics capabilities that Perfecto or Perforce is looking to offer in its roadmap, it will certainly help if they also cater to executing and enabling decision-making, rather than just focusing on standard testing metrics such as execution, efficiency, and defect rate. These are good metrics, but they don't necessarily enable decision-making for SLTs. Any improvements in the dashboards and reporting tools should focus on metrics or SLAs that can help with decision-making.
There was a discussion about having the capability to export the test results to a certain tool that we use in our project. If that were added it would be great not having to manually take screenshots, put them in a document, and share them on the different test management tools.
I'm hoping that Perfecto will come up with browser testing as well, because it would be easier to access it.
I would like to see the inclusion of machine learning features. If we can have that, it will be a better tool. On the analytics side, Perfecto is in need of improvement. The product has a cool dashboard, no doubt about that. But that dashboard is limited to visualization and slicing and dicing the numbers, as well as getting the reports in multiple formats. What I would like, beyond that, is if we can have some predictive intelligence capabilities. This includes machine learning and modeling capabilities. I would really like to see that from Perfecto. As an example, I would like to be able to predict our failure rate in an upcoming release of our software. The reliability of devices is very good, at perhaps 99.9%. However, sometimes, we find that some devices are offline and not available. I would like them to achieve even closer to 100% reliability. I know that 100% is not achievable but they can still push that number. The timing of the software is an area that can be improved. For example, there is a new Appium release and Perfecto should be able to get it to us more quickly. They take their own time for internal testing before they roll it out to customers but we would like to use it faster, so it would be helpful if they can increase the speed of that process. It is somewhat related to improving their time to market.
The monitoring features, in particular network traffic monitoring, could be improved.
I can't really say anything negative about Perfect, but if we talk about feature enhancement, there are a couple of things that I'm working on with our customer support manager. I'm hoping they can support on-premises instances. We have been working on a JIRA integration with Perfecto—and I'm extremely impressed that they have that—but at this time they're not supporting onsite JIRA instances, which is what we have. I love the idea that the testers will be able to run either their manual or their automation scripts, and that the test report will automatically go into that team's JIRA instance. That will save some time. The testers won't have to extract the report, save it, and bring it over to their JIRA instances. Then, when the scrum master reviews the test results, if there are questions, they have all of the details from the report right there. One other enhancement that I've already spoken with them about is that they don't support Android for ADA (Americans with Disabilities Act) testing. They mentioned that they're going to try to begin supporting that later this year.
Its performance should be improved. Anything to speed up the user interface would be a great help. We've had a lot of pain with their migration from a product UI that was based on Adobe Flash to the new product that is based on HTML5. Migrations like that seemed to be very painful and not a real smooth process. We're still sort of recovering from that migration from old technology to new, and we haven't got all the functionality ported over that we used to have on the old UI. It is sort of unfair to ask, but we don't have SSO right now. So, we're working towards that. If that had been available from day one, it would have been super awesome. Once we get to that point, it would be a whole lot better. There could be some additional administrative tools and reporting tools to get more direct hands-on reporting for usage, user database, and things like that.
We bought this solution because it is supposed to be proficient at both web and mobile testing. It does well for mobile testing, but when it comes to the web aspect, it is lagging a little bit in terms of execution. Specifically, if it is a native app then it does really well. But, with a hybrid, or it is a native mobile web app, it seems to be lagging.
We have had challenges while using this product. The main problem has to do with OTP. OTP comes on the SIM but there is no SIM card in the cloud, so it makes it a big challenge for us. It is available in development but because we don't have the OTP API exposed, we don't have it in production. We have had some issues with performance, which is something that should be improved. The price of Perfecto Mobile should be lowered.
I looked at the solution a year ago. I'm not sure what functionalities were lacking or need improvement, but I do find it to be a good tool. It would be ideal if there was a complete integration with other test management tools and other applications like HPLM, Micro Focus, or Microsoft Azure.