BlazeMeter vs Sauce Labs comparison

Read 113 Sauce Labs reviews

5,277 Views
4,142 Comparison Views

96% willing to recommend

BlazeMeter

Comparison Buyer's Guide

Download the report

Executive Summary

We performed a comparison between BlazeMeter and Sauce Labs based on real PeerSpot user reviews.

Find out in this report how the two Functional Testing Tools solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.

To learn more, read our detailed BlazeMeter vs. Sauce Labs Report (Updated: October 2024).

BlazeMeter vs. Sauce Labs

October 2024

Download the complete report

Helped 816,406 peers since 2012

Categories and Ranking

BlazeMeter

Ranking in Functional Testing Tools

6th

Ranking in Test Automation Tools

5th

Average Rating

8.2

Reviews Sentiment

7.5

Number of Reviews

Ranking in other categories

Performance Testing Tools (4th), Load Testing Tools (4th)

Sauce Labs

Ranking in Functional Testing Tools

12th

Ranking in Test Automation Tools

15th

Average Rating

8.8

Number of Reviews

113

Ranking in other categories

No ranking in other categories

Mindshare comparison

As of November 2024, in the Functional Testing Tools category, the mindshare of BlazeMeter is 0.7%, up from 0.6% compared to the previous year. The mindshare of Sauce Labs is 6.7%, down from 7.9% compared to the previous year. It is calculated based on PeerSpot user engagement data.

Functional Testing Tools

Featured Reviews

Bala Maddu

Mobile Network Automation Architect at BT - British Telecom

Reduced our test operating costs, provides quick feedback, and helps us understand how to build better test cases

Overall, it's helped our ability to address test data challenges. The test data features on their own are very good, but version control test data isn't included yet. I think that's an area for improvement. We can update the test data on the cloud. That's a good feature. There's also test data management, which is good. [Runscope] doesn't have the test data management yet. Mock services do, and performance testing has it. We can do the same test through JMeter, validating the same criteria, but the feedback from [Runscope] is quite visible. We can see the request and the response, what data comes back, and add the validation criteria. We can manage the test environments and test data, but running the same API request for multiple test data is missing. We cloned the test cases multiple times to run it. They need to work on that. Version controlling of the test cases and the information, the ability to compare the current version and the previous version within [Runscope] would be really nice. The history shows who made the changes, but it doesn't compare the changes. In the future, I would like to see integrations with GitLab and external Git reports so we could have some sort of version control outside as well. There is no current mechanism for that. The ability to have direct imports of spoken API specifications instead of converting them to JSON would be nice. There are some features they could work on.

Read full review

Savio De Souza

Sr. IT Architect at a healthcare company with 10,001+ employees

Enables us to have a fully functional CI/CD process while saving time and cutting down on the training required for individual testers

When we were in development, it was a bit of a pain because we have onshore and offshore development. One of our development shops is in India, and we were running tests over there. When some of the users tried to log in, it was slow for them or we didn't have enough licenses. That was during the core development before we even launched. We got around that by purchasing more seats, tinkering with some of the virtualization pieces, and scaling. Now we don't have that issue, because we scaled back the offshore team significantly, so when we run it overnight, there's really no effect. We come in the next morning and review the results. It doesn't affect the overall business or the offshore team. There were also some bottlenecks because of the amount of time testing takes, so we started using more tunnels and running it in parallel. That was the main issue that we faced initially, but now that it's all set up, we're good to go. We were struggling with the volume of tests, and Sauce Labs suggested we run everything in parallel. Sauce Labs isn't lacking any features that we want, and it has several we're not using, like mobile and API testing. They've also introduced a ton more features since we launched, so I don't see anything missing on their end.

Read full review

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros

"The orchestration feature is the most valuable. It's like the tourist backend component of BlazeMeter. It allows me to essentially give BlazeMeter multiple JMeter scripts and a YAML file, and it will orchestrate and execute that load test and all those scripts as I define them."

"The product's initial setup phase was straightforward."

"The feature that stands out the most is their action groups. They act like functions or methods and code, allowing us to reuse portions of our tests. That also means we have a single point for maintenance when updates are required. Instead of updating a hundred different test cases, we update one action group, and the test cases using that action group will update."

"The user interface is good."

"The solution’s most valuable feature is the dashboard."

"The baseline comparison in BlazeMeter is very easy, especially considering the different tests that users can easily compare."

"The extensibility that the tool offers across environments and teams is valuable."

"In our company, various teams use BlazeMeter, particularly appreciating its cloud license software, which supports up to 5,000 users. BlazeMeter's cloud capabilities allow us to load test or simulate traffic from any location worldwide, such as Europe, North America, South America, Australia, and even specific cities like Delhi. So, with one cloud license, we can simulate user load from various locations globally."

More BlazeMeter pros

"It runs on the cloud, so you don't have physical setups to run all of this."

"Live device testing. As we all know, It's really hard and challenging to find/purchase many real devices to test because it will be costly and not all the team can be able to purchase all of the devices out there. We used to have a lot of real devices under our labs. However, it is really time-consuming to maintain those devices and make sure they are up to date with the testing requirements."

"From an infrastructure support perspective, the number of VMs, browsers installations and versions that we would be maintaining without Sauce Labs would be a lot. This includes not only the infrastructure costs, but also the maintenance costs and people's time. The labor cost associated with maintaining all of that would be considerably high. In terms of efficiency, having concurrent VMs with various browser combinations available has allowed us to run multiple executions by all our teams."

"The most valuable feature is the ability to easily test multiple browsers and versions, as well as off-loading our local environments using the SauceCloud."

"It has significantly enhanced our testing accuracy by approximately 50%."

"The most valuable feature is the cross-browser feature, it has many android and iOS devices both simulators and real devices. It's easy to integrate. I also like video recording too."

"The error logging is also very robust. If we run a test through Sauce Labs and there's some sort of issue, a log will appear on the screen. Log messages are usually heinous and horrible... Sauce Labs is incredibly good at saying things like, 'Hey, here is the exact issue. Fix this and you can run the test.' That helps in getting things up and running and executing the way they should."

"Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. We have an integration with App Center that we're working on. They have a storage API that lets us retrieve APK and IPA, iOS and Android builds off the phone, so that we can continue testing with CI/CD. They integrate with Jenkins, and Jenkins is the main CI/CD."

More Sauce Labs pros

Cons

"The product currently doesn't allow users to run parallel thread groups, making it an area that should be considered for improvement."

"Integration is one of the things lacking in BlazeMeter compared to some newer options."

"The Timeline Report panel has no customization options. One feature that I missed was not having a time filter, which I had in ELK. For example, there are only filter requests for a time of less than 5 seconds."

"My only complaint is about the technical support, where sometimes I found that they would not read into and understand the details of my question before answering it."

"From a performance perspective, BlazeMeter needs to be improved...BlazeMeter has not found the extensions for WebSockets or Java Applet."

"Having more options for customization would be helpful."

"The seamless integration with mobiles could be improved."

"If the solution had better support and the documentation was efficient it would do better in the market."

More BlazeMeter cons

"They should provide a JIRA integration plugin so that we can easily log issues."

"There have been various times throughout the last month or so where the service has gone down during business hours."

"The ability to configure the memory and CPU for the test machines should be included."

"We have faced challenges with the availability of mobile devices. There was once or twice where there were no mobile devices available."

"Speeds up the time it takes to run end-to-end user interface (UI) tests inside their virtual machines (VMs)."

"Lacks the ability to start multiple tests simultaneously."

"We have found that during automated testing this can be very slow. This causes inconsistencies with the tests. It's very difficult to rely on a service when you can't be sure if a test will pass or fail the next time it runs. This means building in a lot of sync time into the tests which in turn slows them down. If this speed could be improved then the service would be much better."

"We encountered minor issues with stability from time to time but Sauce Labs continues to make improvements."

More Sauce Labs cons

Pricing and Cost Advice

"My company has opted for a pay-as-you-go model, so we don't make use of the free version of the product."

"When compared with the cost of the licenses of other tools, BlazeMeter's license price is good."

"It's consumption-based pricing but with a ceiling. They're called CVUs, or consumption variable units. We can use API testing, GUI testing, and test data, but everything gets converted into CVUs, so we are free to use the platform in its entirety without getting bogged down by a license for certain testing areas. We know for sure how much we are going to spend."

"The pricing is manageable. It is not that big. Big companies won't mind the licensing costs."

"I would rate the pricing a three out of ten, where one is very cheap, and ten is very expensive."

"The product isn't cheap, but it isn't the most expensive on the market. During our proof of concept, we discovered that you get what you pay for; we found a cheaper solution we tested to be full of bugs. Therefore, we are willing to pay the higher price tag for the quality BlazeMeter offers."

"The solution is free and open source."

"I rate the product's price two on a scale of one to ten, where one is very cheap, and ten is very expensive. The solution is not expensive."

More BlazeMeter pricing and cost advice

"Try the free trial and work with a Sauce Labs representative to see what package works best for your application(s)."

"The number of concurrent VMs that Sauce Labs provides depends on your purchase license level."

"It is an expensive tool."

"We did initially go with Sauce Labs because of the pricing and integration."

"It could be less like pay-per-use with a lower rate."

"We have an enterprise account; it has worked great for our needs."

"We pay for a specific number of VMs."

"They could improve on the pricing because it seems pretty expensive. I'm sure it's justified, but it's expensive."

More Sauce Labs pricing and cost advice

See which vendors are best for you

Use our free recommendation engine to learn which Functional Testing Tools solutions are best for your needs.

See recommendations

816,406 professionals have used our research since 2012.

Top Industries

By visitors reading reviews

Financial Services Firm

21%

Computer Software Company

17%

Manufacturing Company

Retailer

Financial Services Firm

18%

Computer Software Company

17%

Manufacturing Company

Healthcare Company

Company Size

By reviewers

Large Enterprise

Midsize Enterprise

Small Business

Questions from the Community

How does BlazeMeter compare with Apache JMeter?

Blazemeter is a continuous testing platform that provides scriptless test automation. It unifies functional and performance testing, enabling users to monitor and test public and private APIs. We ...

What do you like most about BlazeMeter?

It has a unique programming dashboard that is very user-friendly.

What is your experience regarding pricing and costs for BlazeMeter?

BlazeMeter's pricing is competitive but can be negotiable.

What do you like most about Sauce Labs?

It has significantly enhanced our testing accuracy by approximately 50%.

What needs improvement with Sauce Labs?

Sauce Labs can include new technologies like generative AI, which can reduce the human effort in writing test cases. For example, in my current project, we reduced the time it took to complete user...

What is your primary use case for Sauce Labs?

I work as an automation engineer using Selenium WebDriver with Java, and API automation using Rest Assured with Java. I have also worked with Docker integration on AWS. Additionally, I have experie...

Apache JMeter vs BlazeMeter

Comparisons

Compared 68% of the time

OpenText LoadRunner Cloud vs BlazeMeter

Compared 9% of the time

Tricentis NeoLoad vs BlazeMeter

Compared 9% of the time

OpenText LoadRunner Professional vs BlazeMeter

Compared 3% of the time

More BlazeMeter Competitors

BrowserStack vs Sauce Labs

Compared 46% of the time

Perfecto vs Sauce Labs

Compared 23% of the time

LambdaTest vs Sauce Labs

Compared 13% of the time

Bitbar vs Sauce Labs

Compared 3% of the time

Eggplant Test vs Sauce Labs

Compared 1% of the time

More Sauce Labs Competitors

Product Reports

Download BlazeMeter product report

BlazeMeter

November 2024

Download Sauce Labs product report

October 2024

Also Known As

JMeter Cloud

No data available

Learn More

Perforce

Overview

BlazeMeter ensures delivery of high-performance software by enabling DevOps teams to quickly and easily run open-source-based performance tests against any mobile app, website or API at massive scale to validate performance at every stage of software delivery.

The rapidly growing BlazeMeter community has more than 100,000 developers and includes prominent global brands such as Adobe, Atlassian, Gap, NBC Universal, Pfizer and Walmart as customers. Founded in 2011, the company is headquartered in Palo Alto, Calif., with its research and development in Tel Aviv.

Sauce Labs is a functional testing tool that ensures your apps and websites work flawlessly on every browser, OS, and device. The solution allows you to automate functional testing on multiple operating systems and browsers, emulating the way that a user would use the website. With Sauce Labs, you can also run tests on various operating system and browser combinations in parallel, reducing the amount of time to get results. The Sauce Labs solution provides enterprise-grade security, scalability, and reliability.

Sauce Labs Features

Sauce Labs has many valuable key features. Some of the most useful ones include:

Automated cross-browser testing
Mobile testing
Secure tunneling protocol
Usage reporting
Enterprise security
Provisioning
Centralized user account management
Access controls
Support for Selenium, Appium, and JUnit Testing

Sauce Labs Benefits

There are many benefits to implementing Sauce Labs. Some of the biggest advantages the solution offers include:

Speed up development time: The solution makes it possible for you to test web and mobile apps in parallel and accelerate your testing time by up to 10x or more. Using Sauce Labs, you can increase developer productivity because your teams will be able to spend less time with test maintenance and debug faster.

No worrying about infrastructure: With Sauce Labs, you don’t have to buy or maintain more servers or mobile devices. You also don’t need to worry about updating browsers or VM licenses. Sauce Labs allows you to easily scale your tests with parallelization to speed up test cycles and boost capacity on their reliable cloud.

Run tests securely: The solution enables you to securely run your scripts and access your data and files from behind your firewall. Sauce Labs generates single-use VMs for every test and destroys them immediately afterwards so your data always remains secure.

Easy integration for your CI systems: Sauce Labs automated testing platform integrates with Jenkins, Bamboo, Microsoft VSTS, and others so you can test as fast as you develop. In addition, you can share results instantly by using the solution’s simple plugins.

Reviews from Real Users

Sauce Labs is a solution that stands out when compared to many of its competitors. It has valuable features that include error logging, helpful dashboards, and its browser.

Joel A., Product Manager - Data & APIs at a marketing services firm, says, "The error logging is very robust. If we run a test through Sauce Labs and there's some sort of issue, a log will appear on the screen. Log messages are usually heinous and horrible... Sauce Labs is incredibly good at saying things like, 'Hey, here is the exact issue. Fix this and you can run the test.' That helps in getting things up and running and executing the way they should."

Savio D., Sr. IT Architect at a healthcare company, mentions, "Sauce Labs' dashboards contain multiple useful metrics in one place. Everything is represented to us visually on the dashboard, which helps us understand where to focus our attention, what the issues are, and what we need to resolve."

PeerSpot reviewer, Rob L., Director of Quality Assurance - Shared Service at a financial services firm, writes, “The most valuable feature for us is the browser. The most critical thing is that this software aligns with our Agile and DevOps way of doing things. It integrates with kickoff scripts through DevOps.”

Sample Customers

DIRECTV, GAP, MIT, NBCUniversal, Pfizer, StubHub

Salesforce.com, Mozilla, Zendesk, Puppet Labs, Twitter, Bank of America, Eventbrite, Bleacher Report, Okta, Intuit, Travelocity, Sharecare, CapitalOne.