Try our new research platform with insights from 80,000+ expert users

HeadSpin vs Perfecto comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Dec 18, 2024
 

Categories and Ranking

HeadSpin
Ranking in Functional Testing Tools
29th
Ranking in Mobile App Testing Tools
9th
Average Rating
8.0
Reviews Sentiment
6.8
Number of Reviews
6
Ranking in other categories
Mobile APM (8th)
Perfecto
Ranking in Functional Testing Tools
11th
Ranking in Mobile App Testing Tools
4th
Average Rating
8.4
Reviews Sentiment
7.4
Number of Reviews
23
Ranking in other categories
Performance Testing Tools (13th), Test Automation Tools (13th)
 

Mindshare comparison

As of December 2024, in the Mobile App Testing Tools category, the mindshare of HeadSpin is 2.7%, down from 3.1% compared to the previous year. The mindshare of Perfecto is 7.4%, down from 8.3% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Mobile App Testing Tools
 

Featured Reviews

Saorabh Singh - PeerSpot reviewer
It fulfills everything from automation to manual performance
The most valuable features of the product are the performance parameters it gives us, as well as the seamless connectivity with our automation suites. I am also pleased with the continuous enhancements made to HeadSpin. There have been many features added since we started using the product, and all of them are useful.
Roland Castelino - PeerSpot reviewer
Its reporting allows us to have a clear view regarding what tests have been executed
The most valuable would be their Live Stream analysis, where I can see the live analysis of all the executions on a single device or multiple devices as well as track them. The live analysis and reporting would be the single most valuable feature. We leverage Perfecto’s reporting and analytics a lot. From the CI Dashboard, it is mainly the status, which is the past, failure count, and time consumption, e.g., how much time did an average test or script take? Along with that, it provides the historical view compared to the previous result, e.g., am I a pass or fail? Also, the stack trace is very important. Whenever a pass occurs, we don't look beyond that. However, whenever a failure occurs, the stack trace information that it gives us is pretty critical for us when figuring out where failures lie. It gives a summary for the pass/fail count, total test count, the historical view, time consumption for each test as well as the total tests, and the stack rate of the failure. Perfecto's analytics are very important since we use them on a daily basis. We run our executions daily. After every execution, we pull information from the Perfecto reporting system and share that with our stakeholders. Having this information accurately reported is pretty important for us, so everybody is aware of the current status of the product. That way, we can evaluate the health of the product or environment against that which has been executed. Therefore, it helps make those real-time decisions and highlights the impact to the business. I found Perfecto to be pretty easy to use while executing against cross-platforms. The main reason is because the same script or test automation where we execute on multiple platforms has minimal changes that I need to do. Also, it is easy for me to set up an execution on one platform, then on another platform, either in parallel or one after the other. Parallel opportunities save me time. Once the execution has been completed across these different configurations, I can always check and compare, e.g., what are the differences and consistencies? We utilize Perfecto’s cloud-based lab to test across devices, browsers, and OSs. I use it occasionally for manual testing. Though, there are other team members who use it more frequently than I do. I use it mainly for executing my automated tests. We have the Perfecto lab, cloud devices, and machines. I can program my test to execute against any of those devices, which gives me more confidence in my product. I can compare and see how my product or application functionally behaves across these different devices and from a UI point of view, which helps me a lot. The device lab is extremely important to our testing operations. We rely on having multiple devices up and running all the time. Whenever we kick off an execution, there are multiple reasons why executions may get triggered: * CodeCommit * A scheduled job. * Might be on-demand by any stakeholder. We need the lab to be available, as we need devices up and running for executions to take place. Also, the devices help since they allow us to have parallel execution, and not just wait for a sequential device to become free and available. Therefore, volume is definitely key. It also gives us an opportunity to compare execution across platforms in that space. It is extremely important to you that the lab provides same-day access to new devices since we analyze that data every single day after execution. Perfecto provides their own framework called Quantum Framework. That is one option. The other option is, if I want to have my own framework, I can have a Java-based Maven project, take a Selenium library, AppiumLibrary, and REST Assured library, and utilize the open-source framework. It is easy for us to connect to Perfecto, no matter what framework we use, as long as it has these core libraries in it. I can design and structure it any way that I want. The execution will happen in Perfecto no matter what since they have support for these tools or libraries. It is pretty neat that way. We are not dependent on using just one particular framework to use Perfecto. While there are still some framework limitations, there is the opportunity to use multiple, different open-source frameworks, then pass the execution to Perfecto. We can use most frameworks, then design and craft it any way that we want, then just pass the execution to Perfecto.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"The most valuable features of the product are the performance parameters it gives us."
"The most valuable feature is that this is the first connected intelligence all-in-one platform."
"The technical support is really helpful because we can set up direct calls with them if we want to. We can use Zoom or Google Meet to interact with them directly, and if there is an issue in our system, they will help us by reproducing the issue in their machines and trying to figure out a solution. The support is really smooth, and we like that they're very supportive."
"The initial setup of HeadSpin was very easy and user-friendly. It was easy to configure and write a script."
"It has an interesting feature called AV box testing. A lot of companies that are in the OTT segment don't really understand what their streaming is like. They can't test for streaming quality. There are restrictions where you cannot simulate live streaming. For example, on Netflix, you can't simulate how a movie is being streamed on a remote device. That's why HeadSpin has got this AV box testing feature. It is a patented feature. They send an AV box to your location, and you can test live streaming, which is something that no other company does."
"The most valuable feature of HeadSpin it's the integration with other solutions. It is great. I can search for an element or do a quick debugging on the application right on HeadSpin. It's very useful."
"There are a whole bunch of things that I like about the solution, but I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. The reports are very detailed."
"The CI dashboard tool is very good, as is the Live Stream monitoring. Whenever I want to monitor execution, I can open multiple tabs in Perfecto and it is easy for me to refer to the CI dashboard and the Live Stream."
"Mobile testing is the most valuable feature as it has reduced dependency on physical devices. We are located offshore and we don't have the physical devices, and shipping physical devices after every new release would be a difficult task. But with Perfecto, it is easy."
"I also like the reporting functions. We are constantly downloading these reports and sharing them with our final customers. They help us understand what kind of bugs are happening through the applications. The recording feature is handy because it lets us see a video of the process we run through the pipeline and discover the point at which the automation is breaking."
"Perfecto has affected our software quality in a good way. It has allowed us to execute on-demand and on-choice. We also track the number of issues that we find in the product. Every single day, we tag the issues that we found. For example, if something was found by automation, that means it was found by a Perfecto execution. Over time, we realized the real value in tracking those numbers. We can see now that we have clearly been finding issues earlier. It has allowed us to catch our defects earlier, thus improving the quality of our applications."
"The quality of our software has improved since we implemented this solution."
"It creates a faster production cycle and is quick to market. Things get deployed earlier because the testing happens on time. We can do a lot of panelization, so a lot of test phases can happen in a panel. People don't have to wait for a device to come to them. They can access multiple devices at the same time and do testing at the same time."
"The automation piece is the most valuable feature. Every time we had a new version of either OS or an application, we found that being able to automate the testing across different devices is very valuable."
 

Cons

"HeadSpin could improve on the user interface because it is very poor. The checks that are done on the iOS devices are very difficult, but for Android, it runs great. For all iOS devices, the user interface and how it interacts with the device are very poor."
"HeadSpin needs to improve the hardware. With the mobile, the battery life reduces and must be continuously charged."
"They should automate their onboarding. A lot of things are still manual. They can create a video assistant or something like that to completely automate the entire process."
"If you want to do some testing or check the devices manually or check the application in a particular device manually, it is really laggy. That's a disappointment because sometimes we would like to do manual testing when our local devices are not available."
"Support and pricing could be improved."
"Sometimes, devices go offline and some features are not functioning on some devices, specifically on iOS."
"When using devices on the cloud, it lags quite a bit at times. I know that these are real devices that are being projected on our laptop screens and monitors, but if the speed could be improved, that would be good."
"It would be nice if there were some kind of AI to compile a list of available devices. Currently, we have to look at the web interface to see the available devices, but the pipelines can't do it on their own there. We always need to do this manually, so it would be better if this feature were automated."
"Previously, we used the cradle. Every time the mobile was blocking it, we would have to ask Perfecto to provide another one. That took a lot of time away from us."
"It would be ideal if there was a complete integration with other test management tools and other applications like HPLM, Micro Focus, or Microsoft Azure."
"If we could run an accessibility test in Perfecto against builds, it would help us a lot. Currently, that's a very manual process for us. We haven't found a tool that can do accessibility scans for iOS and doesn't depend on engineering effort. Having a feature related to that would be really awesome for us."
"It does well for mobile testing, but when it comes to the web aspect, it is lagging a little bit in terms of execution."
"I'm hoping they can support on-premises instances. We have been working on a JIRA integration with Perfecto—and I'm extremely impressed that they have that—but at this time they're not supporting onsite JIRA instances, which is what we have."
"We've had a couple of issues lately with videos not loading or browsers dying after some execution, although that happens very rarely."
 

Pricing and Cost Advice

"I believe the licensing cost is cheap because it's a total solution, hardware, license and software."
"We have a yearly license for 16 devices."
"It has a yearly license. There is no other option. It is expensive. There are a lot of other cheaper players in the market, but it is like a Mercedes. You pay an extra premium for it, but you get the benefits. I would love to see them come up with project-based costing. Companies that are low on funds or new-age can do with pricing that is easily digestible. They can give them a pricing model for three months. They can provide a startup package."
"It's not cheap, but there are a few different packages and different prices for enterprises with different product versions."
"Pricing is an area where Perfecto can do a little better. When we obtain additional licenses, we enter into negotiations with them."
"I am not sure about its pricing, but from our perspective, licensing has been easy. Anytime I have new users or requests for users that want to get added, it's a very simple process. I just give the architectural owner of the product the name and email address, and they're able to easily add a new user. We don't have any issues in regards to getting licenses, but I don't have any insights into pricing."
"Perfecto's price is excellent compared to other products with similar features. It was the lowest of the three we evaluated. We also established a partnership with Perfecto, so they provide discounts when we sell Perfecto projects and licenses to our customers."
"Perfecto has definitely saved us on the costs and efforts of having to maintain our own virtual test environment. We lost about 20 devices in the past to maintenance and audit. That was a massive loss for us, as a company, because we were giving devices to someone, but don't know whether we would get it back or not. Having those virtual labs, we don't need to worry about these kinds of things. We are easily saving $5,000 to $10,000 a month on device costing."
"Perfecto is about 30-40% cheaper than Device Anywhere. That was a big reason why we switched. Perfecto also solves some of the issues that we had with Device Anywhere. We have grown by 100% since we started to use Perfecto, and now we have devices roaming. When we look at the competition, we would still stick with Perfecto."
"It's definitely on the higher end of prices for this type of service."
"This is an expensive solution compared to others, by 30% to 40%."
"Pricing-wise, it is fine. It is not as expensive as what we used to have in the past from HP, IBM, and others. It is decently priced."
report
Use our free recommendation engine to learn which Mobile App Testing Tools solutions are best for your needs.
824,053 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Manufacturing Company
17%
Computer Software Company
14%
Financial Services Firm
13%
Legal Firm
8%
Financial Services Firm
20%
Computer Software Company
17%
Manufacturing Company
9%
Healthcare Company
5%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
No data available
 

Comparisons

 

Also Known As

No data available
Perfecto Mobile, Perfecto Web
 

Learn More

 

Overview

 

Sample Customers

Zynga, Tinder, Pinterest, Akamai, Microsoft, Airbnb, Jam City, TMobile, Mozilla, CNN, Cognizant, Yahoo!, ebay, Quora, Walmart, Kohls, Telstra
Virgin Media, Paychex, Rabobank, R+V, Discover
Find out what your peers are saying about HeadSpin vs. Perfecto and other solutions. Updated: November 2024.
824,053 professionals have used our research since 2012.