Sauce Labs Reviews

Our healthcare company has a QA team who uses Sause Labs to review various metrics.

For me, as a software engineering manager, I am able to see who from my team is logging on, when are they logging on, and other things in regards to activity within the environment.

The platform is optimized for automation and integration with major CICD platforms and developer tools. This allows us to speed up our production without burning up all of our resources.

It also provides access to automated functional testing (this couples nicely with our other functional testing that we have), visual registration testing (which is as easy as a click of a button), and front end performance testing (which provides a clear understanding of our current state of performance).

Sauce Labs provides us with an all-in-one solution that saves us time, money, and human capital.

Since this is an all-in-one testing site, we are able to take advantage of the browser OS combinations, mobile emulators and simulators, and real mobile devices. This is important to us since we have a variety of users, browsers, OS, etc.

I would like for there to be more detail in regards to the quality of our code i.e. how many failures occurred, how many passed based on industry standard metrics, etc.

I have been using this solution for a few months.

This is a stable solution, I have not had any stability issues.

As told by our rep, this is a fairly scalable solution.

I was not a part of the initial setup though I have all the reason to believe that it was straightforward.

You will save a lot of time thanks to this solutions ability to run parallel testing. My team saves about a week and a half on our deployment schedule. Just be sure that you have settled on your design prior to implementation.

We have several healthcare products across the healthcare continuum, and we use the Sauce Labs platform to test our applications across different browser and OS combinations. We also use it to do mobile testing across different mobile devices that we may not have. It's not easy to set up these different configurations, so the cross-browser and cross-OS platform in the cloud gives us the ability to test across these different configurations without having to set them up or maintain them.

At this point, we are trying to focus on API testing.

My role is with a central team that helps other teams. If another team is struggling then we reach out to them and offer assistance. Because of this, I am familiar with how some teams are using the product. If on the other hand, a team is doing well and doesn't reach out, then I don't have any insight into how they are using the solution or how well things are going. 

We've had an event where I worked with the vendor to coordinate an automation day at my company. They worked with our QA engineers through a training process where we had a test repo that we were able to run across different browser and OS combinations using the platform.

Technical support is very important because it helps us get through issues faster. When we are trying to integrate with a new tool, as is the case now because we're trying to move to GCP, we have a dedicated solution engineer to work with us. The engineer is helping us as we ramp up with GCP, getting our automation to work.

We went through this a few years ago with Azure. Learning a new tool is challenging enough, and then trying to make it work with our existing processes is even more difficult. For example, we need to enable our test automation to run on the new platform and have everything work together. It can be challenging, not to mention that mobile testing is still evolving. This is why support is important. They have helped us with our questions and to get through any blockers.

For the most part, the product is optimized for automation and integration with the major CI/CD platforms and developer tools. That said, our constant complaint is that it is kind of slow. We are trying to figure out ways that we can work around the slowness, and what things we can do better.

We have a lot of products here in the company, and not everyone is leveraging the cross-browser testing platform that is available at Sauce Labs, but we have more and more teams adopting it in the last three years. We are trying to collect more insights into how our users are accessing our applications and using that insight to cover all those different testing configurations using the Sauce Labs platform, so we are trying to leverage the platform more in our internal testing.

For now, Sauce labs covers our needs with respect to the number of browser/OS combinations and mobile devices. However, having more availability and looking into different strategies for making things faster are important. Our goal is always to get faster feedback on a build, which means if it takes a long time to run an automation suite then it's less likely that we'll use it. We're always seeking ways to add capabilities and expedite things.

Having the ability to test our application on additional browser/OS/mobile device combinations opens up additional revenue streams for us because we have a larger user base. It gives us the confidence that the application we are putting out will generate a positive user experience because we've covered all of those testing combinations.

We've gone through a lot of OKRs and have had a general goal to reduce the time it takes for us to test applications. This product enables us to run tests in parallel, which helps to make things faster. I don't have any concrete or documented examples of how long it would take to complete our testing before using the tool because we've evolved the way we write automation.

There were times where it would take half a day to run an entire suite of tests, and through some of the improvements we've made, with and without the tool, our goal has been to be able to run a smoke test in 30 minutes. Some teams are able to achieve that, whereas others cannot.

There is really no one answer that fits because we have teams that are across the spectrum. Some are working with mature products and some are just getting started. I know of one of our teams that optimized their automation and with Sauce Labs, they have been able to take their automated test time down from three hours to approximately 30 minutes. This is only one team out of many but it demonstrates a significant change.

The most valuable feature is cross-browser, cross-OS, cross-mobile device testing.

The parallelization of automated tests is a best practice that I recommend. Some of the teams are able to do that, although some cannot because of the way the tests are formatted. If their configuration allows for it then it is definitely something that I recommend they take advantage of.

Running testing in parallel has worked well for some teams. In one instance, a team was able to split their tests so that they ran across two platforms. Running these two configurations, they initially cut the three-hour testing time in half. Within each of those configurations, we were able to parallelize even further within our test framework, and ultimately, we were able to get the test time down to 33 minutes.

The process can be a little slow to configure and get started. It's a cloud platform and you're trying to run automated tests. It first has to be configured, then the test downloaded, then you have to configure the environment that you're going to run on, then kick off the test based on what your needs are. All of this takes time.

We have faced challenges with the availability of mobile devices. There was once or twice where there were no mobile devices available. These instances may have been due to a service outage.

I have been using Sauce Labs for approximately five years.

We run our testing on a daily basis and I don't think we've run into any stability issues. Nothing that I am aware of has been a concern.

As we run into issues with anything that we are trying to do or integrating with to run our test automation, we reach out to their support team. Anytime that we need something and we need a quick answer, my first recommendation to anyone that reaches out to us for help is to call support and get it logged. This way, we can get someone to start responding.

Some of the support team members are better than others. Overall, I would rate the support between six and seven out of ten.

We did not use another similar solution prior to Sauce Labs.

Also, personally, I have not used any other cross-browser testing platforms.

I was not involved in the initial setup or getting it integrated with our automation test framework, but I think it was fairly easy.

The initial setup was done in-house.

As a SaaS product, it is always up to date.

The pricing is definitely on the higher end, and there are other options that are more cost-effective. One of our teams that was using Sauce Labs separately decided to go with a different solution that was less costly.

Prior to choosing Sauce Labs, the company evaluated BrowserStack. I was not part of the decision process but the differentiating factor may have been the support.

We have our internal framework and we basically use Sauce Labs for cross-browser capabilities and mobile device capabilities. Since we have our own framework, we have not explored any functional testing capabilities that the platform has.

The vendor has acquired API Fortress, and it is available within our account for free to try out. That is something we're going to have a demo next week to look into because, at this point, we want to start focusing on API testing. It's helpful that we are able to run our UI tests across different platforms but we would like to focus more on the other layers of the application. We would like to test under the hood, the business layer, more.

I would rate this solution an eight out of ten.

We use Sauce Labs for mostly our automated testing on cross-browser and emulator devices. Our team mainly focuses on web product testing so using a third party vendor to help with the external resouces is a must. There are many cases that we need to run our test on multiple browsers like Chrome, Firefox, IE, Edge etc. Sauce Labs has all the options for us. Of course, because our products are mostly web based, we need to ensure the cross-browser testing for every release cycle. Instead of doing it manually on many real devices, we use Sauce Labs since they provide many emulator devices.

Sauce Labs has been helping us a lot in reducing the number of manual testing. By moving most of our test to the cloud and not running on the EC2 instance, we not only save time for the service but also the maintenance for the external servers. Sauce Labs is the best out there. They beat their competitors because of its price, customer support, and an amazing forum where you can ask and get answered really quickly. They never stop expanding. Whenever we have a new update for any browser, devices, etc. they make sure it will be in their devices and browsers farm so users can use it immediately.

Live device testing. As we all know, It's really hard and challenging to find/purchase many real devices to test because it will be costly and not all the team can be able to purchase all of the devices out there. We used to have a lot of real devices under our labs. However, it is really time-consuming to maintain those devices and make sure they are up to date with the testing requirements. Sauce Labs knows that and already has a solution for their live device testing which allows us to test our app on the actual devices out there and see the real result.

As a web product QA team, we sometimes need to spot check some new child site on multiple browsers and OS(es). It was a little time consuming for us since we need to click on each of the browser/OS combinations and start a new session to test. Every sprint, with new features and child pages being added, we mostly need to do the same steps over and over again. We would like to see if Sauce Labs can be able to open multiple different browsers on the same website at the same time, just by 1 click. It will reduce a lot of time for manually click and change the devices/OS combination.

2+ years.

We rarely had any downtime with Saucelabs, except sometimes our tests cannot connect through the tunnel. It's not always but once in a while and with some random tests only. Their support is really great, they always help us to diagnose and find the root cause of the problem. 

The performance is great, we never had any slowness or trouble with running our tests.

Amazing! They are very quick in response and very throughout. They even go way beyond to help troubleshoot our code and give the coding example to help with all the questions that we had.

We did use another third party vendor for cloud testing. However, their price increased a lot, and their support was a little slower than Sauce Labs. That's why we switched.

It was very easy. They have a wiki and a GitHub repo with all of the codes for every programming language. We can either use their template page or inherit it to our use cases.

We developed our test framework based on Sauce Labs in-house. There was no other vendor team.

We have a team that manages all the licenses, and they share the pool with the rest of us. However, we haven't had any problem of running out of concurrent devices/connection because we agree to not running many test in parallel at the same time.

There are only two big companies in this field and since we were moving away from the other one, we obviously did not consider any other options.

If you are looking for a cloud devices testing provider, look no further. They will beat their competitors on price, support, and a lot more. 

There are a few features that makes this service a top-tier solution. First there is the automatic video recording of the test runs that records the actions taken. For any debugging that you have to do this makes this service invaluable. Often times I would look at the recording first before I started debugging the test locally since I would likely find the issue there faster than starting up my own debugging session. 

Second best feature is the use of real mobile devices as an option for some of the test runs. There is no substitute for running tests on real devices. After setting up and maintaining a local testing grid of 9 mobile devices I can assure you using a service that does it for you is so much better. There were some weeks that were spent on just fixing issues when using my own devices.

The best feature though is the fact that they use the WebDriver protocol for executing actions on their service. A lot of companies just used their own variations on the WebDriver. This then meant that you had to write another set of wrapper functions just to make the current framework work with whatever vendor you were using. Sauce Labs just used the standard remote WebDriver. Which meant that all you needed to change was the hub url to point to their hub and you were done. If you wanted to there is a few lines of code that let you mark the test runs as successful or failed from your side instead of their generic completed message. All told that is 5 minutes of work and it would have better integration with whatever your process is.

The ability to share the tests that failed with the developers improved the overall efficiency of the team since there was no ambiguity on why a test failed. In addition we used some of the recorded test runs with the business to clarify the functionality of some of the more complicated features.

My company wanted test coverage for some devices that were not available on Sauce Labs so I still had to manage a small "test lab" with those devices. Some of the devices were Windows Phone 8 and Blackberry 7 and 10. On our site the only reported instances of those devices accessing our site was the test devices after 6 months so I am still unsure why we wanted to support those.

Actually since their service was so stable it helped identify some flaky tests. Before those tests were executed on bad machines so if they failed they would be executed again and then it would pass. But by running the tests on Sauce helped identified that the test itself was flaky so the tests were refactored into stability.

We did not encounter any issues with scalability. I personally wished that we had a budget to get more parallel executions but the cost was not justifiable.

Their customer service reps are fantastic! They have heard of all of the typical issues that can occur in business and offer solutions to it such as using one of the professional tiers to make sure that their service integrates into the workflow before going to the full blown enterprise plans.

We had used Perfecto Mobile but was unhappy since it was not easy to integrate with our existing framework. Business wanted the devices to be real devices everywhere in the world so went ahead and purchased Perfecto Mobile without trying the service.

The initial setup was so easy that I thought I must be missing something. Since their service provides so many features I was certain that there must be some other options that I have to declare in the code to get those features, but no it was really that straight forward. 

I implemented it in-house since I was tired of managing 25 virtual machines and 12 mobile devices and 5 physical machines on a seemingly daily basis.

Truly agile companies that are constantly testing should get the enterprise solution that they have with dedicated machines. For more ad hoc testing or just testing before the release the pricing on their site would be the best.

I evaluated Perfecto Mobile and BrowserStack before testing out Sauce Labs and they did not compete well enough with Sauce Labs. Perfecto did have more devices available around the world than Sauce Labs but the connections were slow.

As we began architecting a CI/CD pipeline in our SDLC, a need quickly arose for left-shifted, continuous testing of our web applications in real browsers. Specifically, a service that could integrate seamlessly with our Jenkins CI servers and executing tests continuously against our Dockerized applications and firewalled environments. Furthermore, our company does not have the resources to implement and maintain a proprietary Selenium Grid for test distribution. Sauce Labs addresses all of these concerns, and furthermore offers a platform for manually debugging applications in any combination of platforms, browsers, and devices needed, either from an automated test script or from a manual test session. The largest selling point for our needs is its relatively seamless integration with our Jenkins CI servers, including detailed playback and test reports per build job, as well as easy configuration of the proxy tunnel used to access our firewalled environments and desired browsers-under-test, right from the Jenkins UI via their Sauce OnDemand Jenkins Plugin.

Prior to moving to a continuous delivery model, we were on a classic two week regression testing cycle, mostly conducted via manual testing and manually-executed automated tests scripts in a large integrated testing environment. Upon shifting our SDLC to be more continuous, and rearchitecting our delivery pipeline, we were able to very quickly shift that long testing cycle to be integrated into the delivery pipeline itself by left-shifting these deep functional/regression tests to be executed during build time on our CI servers, and the implementation of these automated tests scripts being integrated into the development effort itself. Sauce Labs allowed all of this to happen by supplying the service needed to execute automated browser tests in from a CI server without having to implement or maintain our own proprietary grid of virtual machines.

Of all the services explored, Sauce Labs offers the single best solution for integrating deep automated browser testing in a CI/CD pipeline, and integrates best with Jenkins. The Jenkins UI plugin offers a way to easily configure browser settings for tests being executed continuously. The Sauce Connect Proxy is a necessary technology for companies who have firewalled testing environments, as well as localhost testing. Test reports (that also integrate seamlessly with Jenkins jobs) are also detailed and incredibly useful. Browser support is quick to be updated as new versions of browsers are released. The custom capabilities that can be provided to Sauce Labs VMs during automated testing sessions are a valuable option for experimental or niche testing.

The Jenkins Sauce OnDemand plugin could have more options available to tap into more of the custom capabilities Sauce Labs actually supports. Currently, in order to tap into these capabilities, a programmatic solution is required from the test code rather than being able to configure them via the Jenkins Plugin UI, e.g., desktop resolution, browser automation binary versions, remote Selenium version, and browser console logs. These are all things it would be nice to be able to customize straight from the Jenkins Plugin UI. 

In our experience, we have also had some issues with the Sauce Connect Proxy on our Jenkins servers failing to start, which makes the optimal CI/CD pipeline come to a halt. 

User account management needs an overhauls, allowing for user groups rather than just a hierarchy structure. Test job queuing is a first-come-first-serve, meaning there is no inbuilt way to queue batches of jobs from different Jenkins servers/accounts.

Occasionally there are issues with the stability of the Sauce Connect Proxy during Jenkins builds, which can cause builds to fail seemingly inexplicably, since determining the root cause is oftentimes difficult.

The only issue is cost. Each new unit of concurrency you add to your enterprise license represents a relatively linear increase in cost.

When you reach the Enterprise licensing tier, base level being with 10 concurrent test sessions, pricing is essentially per-unit-of-concurrency thereafter with a relatively linear increase and not much benefit for "bulk".

BrowserStack was also explored, but did not integrate with out intended CI/CD model as extensively or seamlessly as Sauce Labs was able to.

The abundance of device, platform, and browser combinations/versions that can be used in parallel.

Test engineers can now run more tests without bogging down the power of their PC which enables them to multitask, getting more work done.

Unable to segregate reports for tests that are currently being developed, and might not be returning useful results.

One year.

Setting up infrastructure was challenging, but worth the effort.

No.

No.

Very positive. From the sales reps to the customer success reps, Sauce Labs is truly dedicated to ensuring an outstanding experience.

Good.

Previously, we had attempted to create our own, on-premise, device grids. There were many difficulties, and in the end, there were cost savings with outsourcing this.

Opening the network securely was challenging, but once established, it was very easy to start executing automated tests through the proxy.

We worked closely with Sauce Labs' support and technical teams to implement the solution properly in-house.

A lot.

Do your homework.

Yes, I won't name them, but we looked at two other products of a similar nature.

The APIs that are offered play a huge role in reporting and metrics for us. Figuring out how to utilize them can be of a tremendous value add to any organization making data-driven decisions.

We use Sauce Labs for the following:

- Executing automated functional tests across multiple applications in a combination of about 5 browsers/os. These include IE11, Safari (latest), Edge (latest), Firefox (latest) and Chrome (latest).

- Running manual exploratory testing across the same browser mentioned above to get a hands-on view of the application running in each of the environments.

- We use the screenshots and videos to share the bugs or issues found with the teams to assist in the resolution of the bugs.

Sauce Labs has improved the testing side of our organization in the following ways:

Sauce Labs runs a huge number of browser and operating system combinations it allows our teams to worry more about testing our applications than maintaining a combination of browser and os. Either through the Selenium grid or running the browsers locally. In essence, it allows our testers to concentrate on testing the application and not worrying about the infrastructure. It also allows us to target new browsers without the required setup of the new browser.

As stated earlier we use Sauce Labs for a combination of automated testing and manual testing. Therefore the most useful features are the ability to run the functional automated tests via Sauce Labs tunnels which allows access to applications in our internal network. The second most useful feature is the manual side. This allows our testers to quickly identify issues with the look and feel of our apps in the different browsers we support.  I have not had a requirement for mobile testing so cannot advise on that side.

We have used Sauce Labs extensively for the last 5 years across automated and manual testing and have found some problems during this time. Firstly as we use Sauce Labs to connect the tunnel to connect to apps in our internal network we have found that during automated testing this can be very slow. This causes inconsistencies with the tests. It's very difficult to rely on a service when you can't be sure if a test will pass or fail the next time it runs. This means building in a lot of sync time into the tests which in turn slows them down.  If this speed could be improved then the service would be much better.

Approximately 5 years.

The pricing is clear on the site - I would say it is more expensive than some alternatives.

The Primary use caes would be the following :

• Using the device farm and also using the ability to use the virtual and real devices for the purposes of software Quality Analysis testing.
• There are projects that sometimes require a variety of devices to perform software testing on and having the repository of devices to choose from that my company might not have greatly assisted with this use.
• Especially in handling massive loads of users. Since there is virtual emulation as well as device farms with actual devices this allows a great number of devices for a team to test with.

Sauce Labs has been a saving grace for our company. More so in recent years if I'm going by my own personal experience.

When you consider the positive plus points relative to having multiple devices, especially with the recent pandemic being under lockdown as well as just a general work from home situation before. Having Sauce Labs farm has been greatly beneficial to many people within the organization and has helped keep the project turn around time on track.

Supplying devices to a testing team of possibly close to one thousand testers and developers is a great undertaking but Sauce Labs has made this very easy and a welcomed solution.

The most valuable feature is as I mentioned would be the benefit of having mobile devices on hand even if it's virtually so in abundance for our testers In my company. 

Another positive is having an up to date device farm and virtual emulation of devices. The time it takes to allocate and purchase devices on the scale it would normally need is completely deducted from the issues we would have to face from a day perspective or even project-wise. On top of that, having to maintain these devices would be another undertaking and require resources that we as a company do not have to concern ourselves with anymore.

If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farms. While this is absolutely better than not having it situation Sauce Labs certainly would benefit as well as we the user would certainly benefit from gaining a software interaction speed boost. Having it more consistent would be greatly appreciated if the long run. This is the only thing affecting the testing time at the moment and can even be seen as a disadvantage in comparison to having an actual physical device in hand.

i have been using saucelabs for about 6 years or so within the company I work for

Stability-wise it's just okay. The speed of this software needs to improve a lot of possible.

Very good as it has the ability to appeal to a lot of users despite some lagful moments.

Customer service is good. I have personally not had an issue myself but from those that have it has never been anything but positive .

I have not personally used another solution prior to Sauce Labs.

Pretty straightforward for the most part.

We deployed with an in-house team.

In terms of projects, it's per-project basis. So it varies but it certainly has saved time.

I'm unaware if my company has done so but I imagine they did and still chose Sauce Labs as the best option.

It's certainly worth a try if anything . I can confirm it has definitely been a help to myself and the company I work for and it probably will do the same for others.