Sauce Labs Reviews

Automated mobile testing dashboard.

We can now run daily and on demand automated tests.

Sometimes the tests can pass or fail inconsistently. The tunnels and jobs timeout before the app loads, or it has to try 3 or 4 times to get a good connection.

4 months.

No.

No.

Not yet :)

I have not used it.

I have not used it.

No, I haven't.

Initial setup was easy. Our company already had the account open, but I simply added the sauce configurations to my test configuration file and ran the test.

With only a few months of service under our teams belt, it has been nothing but excellent. We joined during the web interface transition and it is a more enjoyable experience. Once tapped into their API, our testing suites contain reliable information we can validate across the platform, switching between Sauce results and our own. This obviously makes troubleshooting much less daunting. Their documentation system is much improved. We rely on their services for mobile iOS and web testing across several projects.

With the rebranding and a focus on providing physical device testing. Sauce seems capable of being a top service provider now and into the future.

We run eight virtual iOS simulators with our suite of over 280 feature tests. Parallel runs reduce our total execution runtime to a little over 2 hours. By adding some clever logic into our driver class for Sauce, we can send tests to random device configurations to achieve greater coverage without running the entire suite against every device. Interacting with the API makes this easily possible. We don't need to use Sauce Connect and perform each session configuration within our driver class.

The only caveat to mention, is the availability of the next release of iOS on their systems. Their adoption seems a bit slow. I understand Sauce is making an effort to enhance their current infrastructure to make updating to the latest iOS release easier in the future.

Been using Sauce Labs for a couple years now, and to say the least it's a lot better than managing my own farm of VM's.  Having the capability to run in parallel against various types of OS's/browsers/mobile-devices is great as well.  Looking forward to glancing at RDC (Real Device Cloud) and seeing if that provides a better solution as we run our mobile tests on slow emulators.

Sauce Labs, for the most part, has very little to no down time, so our tests are running in parallel all through out the day and night.  It's great to go back and view the videos of our browser tests to first see what is going on, and then search the log and output files provided to narrow down the issue even further.

I think their logging could be improved a bit, possibly to include the output that I see on my command line (print statements, etc.) which help pin-point the error (if one happens) more easily. 

The most valuable feature would have to be the cross-browser testing functionality. Our product specifications say that we support the Big Four browsers – Chrome, Firefox, IE, Safari – latest versions and up to two versions back. When including mobile, that’s 10 different browsers (and browser drivers) to maintain. This would be a near impossible task for our small testing team.

It also gives us the ability to replay our tests back in video form. This makes attempting to debug failed tests not a breeze, but easier. It also helps convincing (stubborn) developers that there is a problem.

We have been able to successfully integrate Sauce Labs into our automated test environment. From that we are able to run our test suite across multiple browsers overnight. This gives us a level of confidence that we previously didn’t have.

The other huge benefit we get from this product is the manual testing powers. When a client tells us that they’re having issues on a particular browser. We’re able to fire up that exact version and reproduce the problem ourselves.

Speed is the key. We’re a business in Australia, so any tests that we perform are having to travel to America and back. This is time that we have to wait on for us to get responses back. In an active business every second counts.

We've been using it for about 12 months.

We've had no issues with deployment.

We've had no issues with instability.

We haven’t had any problems that we couldn’t work around, or there wasn’t any documentation on. The only problem is that hit that point where it’s unfeasible to expand our UI testing framework as the cost to run a shed load of tests is too much for our business.

This has been an area of focus from Sauce Labs. Their customer service is unmatched, and their help documents have all been recently updated.

We have used a local Selenium grid, and still do to write our tests. However, we valued the cross-browser accessibility over speed for our nightly automated test suite.

Initial setup was somewhat difficult originally. That was due to multiple reasons, namely lack of actual programming experience on our behalf, and the documentation at the time was a mountain to climb. Sauce Labs has since updated this documentation, and we have since gained experience.

Our in-house team performed the implementation. For advice, it’s a lot like setting up Selenium Grid. One thing we did was have an Environmental Variable “ISTESTLOCAL”, we use this as a gate to whether or not our tests should run on Sauce Labs or not. This is very helpful in increasing the speed which we can write and debug our own tests.

Also, if using the Bamboo plugin, make sure to check the settings page as there is a list of already in use environmental variables set when Sauce Labs runs.

The biggest things you have to consider is how much testing you’re planning on doing, who is your client base, and what browsers they’re using, and what can’t be tested using API testing.

If you have any issues with the product just shoot the support team an email. They always get back to me promptly with the best solution they have available. Also, spend the time to read the documentation.

If your company is security conscious, as it should be, look into Sauce Connect which creates a secure tunnel between Sauce Labs and yourself.

Being able to test how my websites render on a very wide range of mobile devices. 

I am a novice web-designer really just producing for my own hypnotherapy business. However, Sauce Labs is incredibly easy to use and has enabled me to make some changes so that the sites are (almost) universal across the range of devices and versions.

A couple of hours

None

None

None

Recommendation from the web developer at British Airways.

Easy

Traffic Travis

Loving it already

I find the API to integrate with my script runner invaluable. I can annotate jobs and get results to display in my CI server. For more complicated scripts, the secure tunnel is extremely useful when automating a 'private' application.

Running WebDriver scripts in house for a single, or few browser combinations is easy but once we started to scale up both in terms of browsers and parallel execution maintaining the grid became a pain so we offloaded it to Sauce. Now we have more combinations that we had before and don't have to worry about scaling our script execution ourselves.

Before starting to use Sauce, we used an internal Se Grid running on a mix of local machines and both local and cloud VMs. The upkeep needs to keep the machines patched and healthy became increasingly a burden -- especially with a run of bad luck with AWS saying that instances were being shut down due to 'performance degradation'. 

What I like the most is the cloud running the automation test with the help of a number of devices. I like that we can use both Android and iOS devices for testing.

Manual testing is also so much easier with Sauce Labs. Running cross browser testing is the most useful functionality for me.

With Sauce Labs testing, manual plus automation is very easy. It reduces running test times.

Connectivity needs to increase.

Two years.

No issues.

No issues.

No issues.

Good.

Good.

No.

It was straightforward.

In-house.

Good.

It is a good purchase. The cost is also good.

No.

Use it, and enjoy testing with Sauce Labs.

The ability to run on multiple browsers, simulators, and emulators and the ability to proxy to other locations. Also, it runs on the cloud, so you don't have physical setups to run all of this.

Currently, it's being used to run tests after deployment of some of our services so we know the status of the services. We also use it to run tests on a schedule so we can check the status of the services (if it's passing/failing). Because we can start up more than one VM through it, we can also run the tests in parallel, meaning tests can be completed faster than running it locally on one machine.

Start execution time as each time a set of tests start, it will launch a new VM so it takes a bit of time.

A couple of months.

Just some of the slowness, we are still investigating if this can be improved on our end and if it's network related.

Some browsers doesn't support video playback which we use for some of our tests.

If too many tests are run in parallel using Sauce Connect, sometimes the tests time out. So we had to limit the number of parallel executions.

Pretty good. The customer service guys are responsive, and will respond with a reply in less than a day.

It is also quite good. Some good suggestions are given when questions are asked.

Previously, we were only running it locally.

Pretty straightforward as the documentation is quite extensive.

In-house.

I would say quite high.

Trial the product and see if it suits your needs.

Nope.

I would definitely recommend - the community and customer service are good. The improvements are continuous with regular updates.