Try our new research platform with insights from 80,000+ expert users

BlazeMeter vs Sauce Labs comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Dec 15, 2024
 

Categories and Ranking

BlazeMeter
Ranking in Functional Testing Tools
7th
Ranking in Test Automation Tools
6th
Average Rating
8.2
Reviews Sentiment
6.9
Number of Reviews
47
Ranking in other categories
Performance Testing Tools (4th), Load Testing Tools (4th)
Sauce Labs
Ranking in Functional Testing Tools
10th
Ranking in Test Automation Tools
15th
Average Rating
8.8
Reviews Sentiment
7.2
Number of Reviews
113
Ranking in other categories
No ranking in other categories
 

Mindshare comparison

As of December 2024, in the Functional Testing Tools category, the mindshare of BlazeMeter is 0.7%, up from 0.6% compared to the previous year. The mindshare of Sauce Labs is 6.7%, down from 7.7% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Functional Testing Tools
 

Featured Reviews

Bala Maddu - PeerSpot reviewer
Reduced our test operating costs, provides quick feedback, and helps us understand how to build better test cases
Overall, it's helped our ability to address test data challenges. The test data features on their own are very good, but version control test data isn't included yet. I think that's an area for improvement. We can update the test data on the cloud. That's a good feature. There's also test data management, which is good. [Runscope] doesn't have the test data management yet. Mock services do, and performance testing has it. We can do the same test through JMeter, validating the same criteria, but the feedback from [Runscope] is quite visible. We can see the request and the response, what data comes back, and add the validation criteria. We can manage the test environments and test data, but running the same API request for multiple test data is missing. We cloned the test cases multiple times to run it. They need to work on that. Version controlling of the test cases and the information, the ability to compare the current version and the previous version within [Runscope] would be really nice. The history shows who made the changes, but it doesn't compare the changes. In the future, I would like to see integrations with GitLab and external Git reports so we could have some sort of version control outside as well. There is no current mechanism for that. The ability to have direct imports of spoken API specifications instead of converting them to JSON would be nice. There are some features they could work on.
Savio De Souza - PeerSpot reviewer
Enables us to have a fully functional CI/CD process while saving time and cutting down on the training required for individual testers
When we were in development, it was a bit of a pain because we have onshore and offshore development. One of our development shops is in India, and we were running tests over there. When some of the users tried to log in, it was slow for them or we didn't have enough licenses. That was during the core development before we even launched. We got around that by purchasing more seats, tinkering with some of the virtualization pieces, and scaling. Now we don't have that issue, because we scaled back the offshore team significantly, so when we run it overnight, there's really no effect. We come in the next morning and review the results. It doesn't affect the overall business or the offshore team. There were also some bottlenecks because of the amount of time testing takes, so we started using more tunnels and running it in parallel. That was the main issue that we faced initially, but now that it's all set up, we're good to go. We were struggling with the volume of tests, and Sauce Labs suggested we run everything in parallel. Sauce Labs isn't lacking any features that we want, and it has several we're not using, like mobile and API testing. They've also introduced a ton more features since we launched, so I don't see anything missing on their end.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"The on-the-fly test data improved our testing productivity a lot. The new test data features changed how we test the applications because there are different things we can do. We can use mock data or real data. We can also build data based on different formats."
"It is focused on concurrency testing, which has been especially beneficial for us. Their previous experiences had caused major setbacks."
"The solution’s most valuable feature is the dashboard."
"BlazeMeter can be used for both API and performance testing, it is a multi-facility tool."
"Using cloud-based load generators is highly valuable to us, as we can test from outside our network and increase load generation without having to upscale our hardware as much. The cloud load generator is there when we need it and is the feature we leverage the most."
"The product's initial setup phase was straightforward."
"The solution offers flexibility with its configurations."
"The extensibility that the tool offers across environments and teams is valuable."
"Running tests in parallel."
"As stated earlier we use Sauce Labs for a combination of automated testing and manual testing. Therefore the most useful features are the ability to run the functional automated tests via a Sauce Labs tunnels which allows access to applications in our internal network. The second most useful feature is the manual side."
"Supplying devices to a testing team of possibly close to one thousand testers and developers is a great undertaking but Sauce Labs has made this very easy and a welcomed solution."
"From an infrastructure support perspective, the number of VMs, browsers installations and versions that we would be maintaining without Sauce Labs would be a lot. This includes not only the infrastructure costs, but also the maintenance costs and people's time. The labor cost associated with maintaining all of that would be considerably high. In terms of efficiency, having concurrent VMs with various browser combinations available has allowed us to run multiple executions by all our teams."
"Before implementing Sauce Labs, we tested physical devices that team members had to share. It was more feasible when we were all located in one office, but we couldn't leverage our offshore capacity. With this solution, we can do everything remotely, which is essential now that most of us work from home."
"The Failure Analysis feature is really important for us, one of the most important aspects. What is the root cause? Is it because we have a defect or is it that we have a test case that we need to fix or modify? The Failure Analysis is one of the main functionalities that I am exploring all the time in Sauce Labs... The Failure Analysis helps us to discover which test cases we need to work on."
"There is a huge amount of open source, community provided resources."
"Testing my app on cloud has really helped us with save time and resources to procure various hardware and software, and set those up."
 

Cons

"The scanning capability needs improvement."
"Sometimes, when we execute tests, the results calculated by BlazeMeter, specifically the response times for failed transactions, are incorrect."
"The reporting capabilities could be improved."
"One problem, while we are executing a test, is that it will take some time to download data. Let's say I'm performance testing with a high-end load configuration. It takes a minimum of three minutes or so to start the test itself. That's the bad part of the performance testing... every time I rerun the same test, it is downloaded again... That means I have to wait for three to four minutes again."
"In terms of improvement, I would like to have the ability to customize reports."
"The support could be better."
"The scalability features still need improvement."
"The Timeline Report panel has no customization options. One feature that I missed was not having a time filter, which I had in ELK. For example, there are only filter requests for a time of less than 5 seconds."
"I can't remove team members that have left the organization. I can only set them as inactive. It would be really nice to clean up my data and delete them from the team management."
"When we were in development, it was a bit of a pain because we have onshore and offshore development. One of our development shops is in India, and we were running tests over there. When some of the users tried to log in, it was slow for them or we didn't have enough licenses. That was during the core development before we even launched."
"This product is not super scalable, because you have a very specific number of VMs that you can use."
"Sauce Labs' dashboard could be improved by adding more filters and allowing more customization options. There was one instance where the dashboard on the Sauce Labs UI didn't meet our requirements, so we had to use the Sauce Labs API to create some apps and dashboards on our own. The API endpoints could be a little more robust and customizable."
"If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farm"
"Start execution time as each time a set of tests start, it will launch a new VM so it takes a bit of time."
"Sometimes pasting text while using "text object" does not seem to work, and it slows down testing times quite a bit."
"I would like for there to be more detail in regards to the quality of our code i.e. how many failures occurred, how many passed based on industry standard metrics, etc."
 

Pricing and Cost Advice

"When compared with the cost of the licenses of other tools, BlazeMeter's license price is good."
"The solution is free and open source."
"The licensing fees are billed on a monthly basis and they cost approximately $100 for the basic plan."
"I rate the product's price two on a scale of one to ten, where one is very cheap, and ten is very expensive. The solution is not expensive."
"The pricing is manageable. It is not that big. Big companies won't mind the licensing costs."
"The product pricing is reasonable."
"My company has opted for a pay-as-you-go model, so we don't make use of the free version of the product."
"The product isn't cheap, but it isn't the most expensive on the market. During our proof of concept, we discovered that you get what you pay for; we found a cheaper solution we tested to be full of bugs. Therefore, we are willing to pay the higher price tag for the quality BlazeMeter offers."
"It could be less like pay-per-use with a lower rate."
"Compared with other services, Sauce Labs is a bit expensive."
"Go ahead please. Try it at a smaller scale."
"​Now that we have an enterprise license, we no longer have to worry about minutes each month!​"
"We did initially go with Sauce Labs because of the pricing and integration."
"Cost-wise, it's decent. If you have to get the base version out of it, it's the best solution to go with. As compared to other cloud service providers, the pricing of Sauce Labs is decent."
"Try the free trial and work with a Sauce Labs representative to see what package works best for your application(s)."
"With respect to pricing, they did a bundled discount because we went with Sauce Labs for both mobile and browser. They were very competitive on pricing and provided a bundle discount for us as a larger customer."
report
Use our free recommendation engine to learn which Functional Testing Tools solutions are best for your needs.
824,053 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
21%
Computer Software Company
17%
Manufacturing Company
7%
Retailer
7%
Financial Services Firm
19%
Computer Software Company
16%
Manufacturing Company
6%
Retailer
6%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

How does BlazeMeter compare with Apache JMeter?
Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs. We ...
What do you like most about BlazeMeter?
It has a unique programming dashboard that is very user-friendly.
What is your experience regarding pricing and costs for BlazeMeter?
BlazeMeter's pricing is competitive but can be negotiable.
What do you like most about Sauce Labs?
It has significantly enhanced our testing accuracy by approximately 50%.
What needs improvement with Sauce Labs?
Sauce Labs can include new technologies like generative AI, which can reduce the human effort in writing test cases. For example, in my current project, we reduced the time it took to complete user...
What is your primary use case for Sauce Labs?
I work as an automation engineer using Selenium WebDriver with Java, and API automation using Rest Assured with Java. I have also worked with Docker integration on AWS. Additionally, I have experie...
 

Comparisons

 

Also Known As

JMeter Cloud
No data available
 

Overview

 

Sample Customers

DIRECTV, GAP, MIT, NBCUniversal, Pfizer, StubHub
Salesforce.com, Mozilla, Zendesk, Puppet Labs, Twitter, Bank of America, Eventbrite, Bleacher Report, Okta, Intuit, Travelocity, Sharecare, CapitalOne.
Find out what your peers are saying about BlazeMeter vs. Sauce Labs and other solutions. Updated: November 2024.
824,053 professionals have used our research since 2012.