Try our new research platform with insights from 80,000+ expert users

AWS Device Farm vs HeadSpin comparison

 

Comparison Buyer's Guide

Executive Summary
 

Categories and Ranking

AWS Device Farm
Ranking in Mobile App Testing Tools
5th
Average Rating
6.6
Number of Reviews
2
Ranking in other categories
No ranking in other categories
HeadSpin
Ranking in Mobile App Testing Tools
7th
Average Rating
8.0
Number of Reviews
6
Ranking in other categories
Functional Testing Tools (19th), Mobile APM (6th)
 

Mindshare comparison

As of November 2024, in the Mobile App Testing Tools category, the mindshare of AWS Device Farm is 4.2%, down from 6.8% compared to the previous year. The mindshare of HeadSpin is 2.9%, down from 3.3% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Mobile App Testing Tools
 

Featured Reviews

AshishSingh11 - PeerSpot reviewer
Jun 2, 2023
A stable solution used for malware testing and APM that needs to improve its performance
The setup phase was fine because AWS had given the whole tree structure on their website. So, basically, if we go here and there apart from that tree structure, it does not support it. So they have a very sophisticated tree structure. So that is the only tree structure that we have to follow. Else, there will be problems. I rate the setup process a seven out of ten, with ten being the easiest setup. Since I had to study the solution, it took me a week to properly deploy it. I just followed the instructions given by AWS to deploy the solution. So on their website, whatever the steps they had given, I simply kept on following it.
Saorabh Singh - PeerSpot reviewer
Nov 13, 2022
It fulfills everything from automation to manual performance
We use HeadSpin for automated, manual, and performance testing of our apps. Presently, we have 70 users, including QA, developers, product managers, and the operations team HeadSpin allowed us to manage remote work during Covid, enable continuous automation execution, and facilitate collaboration…

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"What I like best about AWS Device Farm is that it offers actual physical devices that let you do more accurate testing because physical devices depict the live testing scenarios much better as opposed to emulated devices. AWS Device Farm is a pretty nice solution. Because it's an AWS service, you can use the CLI to tie in several steps that can create the pipeline, and run it efficiently. AWS Device Farm also gives you monitoring ability, observability, logging, etc., so I'm pretty satisfied with the solution."
"I rate the stability an eight out of ten."
"The technical support is really helpful because we can set up direct calls with them if we want to. We can use Zoom or Google Meet to interact with them directly, and if there is an issue in our system, they will help us by reproducing the issue in their machines and trying to figure out a solution. The support is really smooth, and we like that they're very supportive."
"The most valuable feature is that this is the first connected intelligence all-in-one platform."
"It has an interesting feature called AV box testing. A lot of companies that are in the OTT segment don't really understand what their streaming is like. They can't test for streaming quality. There are restrictions where you cannot simulate live streaming. For example, on Netflix, you can't simulate how a movie is being streamed on a remote device. That's why HeadSpin has got this AV box testing feature. It is a patented feature. They send an AV box to your location, and you can test live streaming, which is something that no other company does."
"The initial setup of HeadSpin was very easy and user-friendly. It was easy to configure and write a script."
"The most valuable features of the product are the performance parameters it gives us."
"The most valuable feature of HeadSpin it's the integration with other solutions. It is great. I can search for an element or do a quick debugging on the application right on HeadSpin. It's very useful."
 

Cons

"It is slow. It is super slow. Performance is an area that can be improved."
"An area for improvement in AWS Device Farm is that it lacks a lot of features that would tie it in with other AWS services. The solution doesn't have great connectivity with other services offered by AWS, for example, AWS Secrets Manager. This should be improved because a lot of times, that missing functionality hampers the quality and engineering standards in terms of deploying the full AWS suite of services. What I'd like to see in the next version of AWS Device Farm is for it to link better, or have some type of enrollment that would tie it in with other AWS services, such as EventBridge, Lambda, Secrets Manager, and any other new service from AWS."
"Sometimes, devices go offline and some features are not functioning on some devices, specifically on iOS."
"HeadSpin could improve on the user interface because it is very poor. The checks that are done on the iOS devices are very difficult, but for Android, it runs great. For all iOS devices, the user interface and how it interacts with the device are very poor."
"They should automate their onboarding. A lot of things are still manual. They can create a video assistant or something like that to completely automate the entire process."
"Support and pricing could be improved."
"HeadSpin needs to improve the hardware. With the mobile, the battery life reduces and must be continuously charged."
"If you want to do some testing or check the devices manually or check the application in a particular device manually, it is really laggy. That's a disappointment because sometimes we would like to do manual testing when our local devices are not available."
 

Pricing and Cost Advice

"AWS Device Farm is an expensive service overall. You pay per device, and the cost for each device isn't cheap. My company paid for a device slot. It's a yearly subscription for a single device slot, so that's more cost-effective for my company. On a scale of one to five, where one is very expensive and five is very cheap, I'd give AWS Device Farm a two. It's on the more expensive side."
"We have a yearly license for 16 devices."
"It's not cheap, but there are a few different packages and different prices for enterprises with different product versions."
"It has a yearly license. There is no other option. It is expensive. There are a lot of other cheaper players in the market, but it is like a Mercedes. You pay an extra premium for it, but you get the benefits. I would love to see them come up with project-based costing. Companies that are low on funds or new-age can do with pricing that is easily digestible. They can give them a pricing model for three months. They can provide a startup package."
"I believe the licensing cost is cheap because it's a total solution, hardware, license and software."
report
Use our free recommendation engine to learn which Mobile App Testing Tools solutions are best for your needs.
814,763 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
22%
Computer Software Company
14%
Insurance Company
8%
Manufacturing Company
8%
Financial Services Firm
22%
Manufacturing Company
15%
Computer Software Company
14%
Legal Firm
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
No data available
No data available
 

Questions from the Community

What do you like most about AWS Device Farm?
I rate the stability an eight out of ten.
What needs improvement with AWS Device Farm?
It is slow. It is super slow. Performance is an area that can be improved. Whenever we upload our Zip files to that particular platform, you know, it takes quite some time, especially if the file i...
What is your primary use case for AWS Device Farm?
We have used the solution for malware testing and APM.
Ask a question
Earn 20 points
 

Overview

 

Sample Customers

NFL, Etsy, Tableau, Gannett, Miniclip, Allstate, Rainforest, goibibo, mysmartprice, Zillow
Zynga, Tinder, Pinterest, Akamai, Microsoft, Airbnb, Jam City, TMobile, Mozilla, CNN, Cognizant, Yahoo!, ebay, Quora, Walmart, Kohls, Telstra
Find out what your peers are saying about AWS Device Farm vs. HeadSpin and other solutions. Updated: October 2024.
814,763 professionals have used our research since 2012.