Sauce Labs Reviews

Sauce Labs is a SaaS service hosted on the company's cloud. They have a number of cell phones and mobile devices housed on their cloud, and my team can perform tests on these devices from around the world. We have about 50 users right now, including business analysts, testers, automation, QA, developers, product owners, and third-party vendors. 

Sauce Labs helped us get ready to roll out our mobile apps for the first time. Before implementing Sauce Labs, we tested physical devices that team members had to share. It was more feasible when we were all located in one office, but we couldn't leverage our offshore capacity. With this solution, we can do everything remotely, which is essential now that most of us work from home. 

There's no need for other physical devices anymore because everyone's using Sauce Labs. It's efficient and it works. We still have some older devices, but nobody bothers with them anymore because Sauce Labs works just as well.
While it doesn't necessarily help us get the product out faster, it does make us more confident in the quality. Sauce Labs Visual also helps us catch bugs earlier in our development cycle.

You can access Sauce Labs remotely from different locations. This feature has been especially useful during the pandemic because the members of my home team aren't in one location anymore. The ability to remotely access devices has been essential.

Sauce Labs provides a single visual snapshot for visual and functional regressions, which we can use as proof when there are defects. It also helps us leverage our existing tools, especially the automation component. Sauce Labs has broad coverage, allowing us to test both functional and visual aspects of the UI. This is crucial because we need to see how the application displays on the various types of devices out there.

We've been using Sauce Labs for around a year and a half.

Sauce Labs' stability is good. We haven't experienced any downtime.

Sauce Labs' scalability is good. It's possible for us to scale higher, so I don't foresee any issues with that.

Sauce Labs' support has been great. I would rate them 10 out of 10 because they respond quickly when we have a problem, but we seldom have issues.

Positive

Setting up Sauce Labs was straightforward because most of the work was on the vendor's side. I think it only took a few days. We didn't deploy it into our system. Sauce Labs set it up for us. It requires some maintenance but not a dedicated person. It's one person spending around 10 percent of their time on this. We have an admin who adds users, so it's not really maintenance—more like administration. 

We purchased the solution through a reseller, and everything went fine. 

I don't have any hard data, but we have seen a return. Sauce Labs expands the number of tests we can run with our staff. For example, if we were limited to five physical devices, we could only test with five people. Now, our testing is only limited by the number of staff. From that perspective, it's a return on our investment.

The pricing model of Sauce Labs could be more flexible. Sauce Labs has just one price for the type of solution and a set number of devices. Other solutions have a fee for the base solution and an additional cost per device. 

If you're a smaller organization, you have to consider your needs. Some smaller companies still need to test various devices, so my advice is to start small and scale up as needed. We had initially planned to start big, but that would have been a big waste.

Eventually, I got approved to expand, but it didn't make sense, so I would advise starting smaller. Everything is about costs. Structure it in different phases. If this is something new for you, start with fewer devices to see if the solution is the right for you because it's costly.

You always have to weigh how much the solution costs versus buying physical devices, so I would say go for a phased approach. That way, you can determine whether it pays for itself. We did a phased approach, and we didn't have to go any higher for at least a year. We might go a little higher based on what we're doing this year. We're still evaluating that right now, so we'll see. 

I don't remember the names of the other solutions we evaluated, but we were going through resellers, and those resellers were using other companies. 

I rate Sauce Labs nine out of 10. If you're thinking about implementing the solution, you should talk to Sauce Labs about your needs and challenges. Their salespeople have a technical background, so they have enough knowledge to help you determine if this solution is right for you.

My company is quite large. My team supports the various different IT teams around the business who then work with their business partners. For example, if you think about our research and development division, our clinical trials division, and our manufacturing division, each one of those business groups has their own IT teams who build tools that the business team needs to do its jobs. My team provides the tools that the IT teams would need to build the tools for their teams. So, we are two or three layers removed from the broad side of the business.

We are the team who provides this solution to other groups. However, in terms of our usage, because the company has done things by paper signatures, official paper, and hand documentation for a long time, there has not been a whole lot of progress yet on the automated testing sector. Therefore, our usage is very small compared to other businesses of our size. 

We have 25 concurrent VM licenses for Sauce Labs today. That provision amount has not changed in my time here because it has never exceeded that capacity. So, it is a slow, upward trend, but it is very slow right now.

In January of 2021, there were 515 automated tests that ran through Sauce Labs in the company. By October, we were just shy of 3,000 tests. That is a 600% increase.

In general, I know that there are different pockets of the business that build apps for different cases. For example, we have an iOS dev team in one part of the business. We have a Windows team in a different part of the business, and they are able to use the same platform without us having to try to support multiple different things. In that sense, the ability to test on additional browser, OS, and mobile device combinations has been a great help. 

All our automated tests run in parallel. This helps us to demonstrate value to other parts of the business who have not yet adopted the solution. For example, when there are groups who use Sauce Labs and we try to share learnings, that team presents, and says, "I have saved this many specific hours. Or, I can run 225 tests (or however many tests) in parallel. It saves us this much time and this many dollars to have our application go out that much faster."

Because there is not super high usage for our team, the cross-browser functionality is a big feature for us. Our machines are mostly Windows. Being able to test with Safari, on a Mac, and other types of browser pieces without having to manage all the infrastructure is the biggest feature that our team enjoys.

Sauce Labs is excellent in terms of the number of browser/OS combinations, mobile emulators and simulators, and real mobile devices that it offers. I have not heard any complaints from my customer teams about any lack of selection or particular combination that they are going after that Sauce Labs doesn't already have.

Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. Their API for inserting test results is fairly robust. This was an important factor for us because our adoption is still kind of low. There are not a lot of teams who know how to use it right out the gate. Now that our adoption is slowly growing and teams are figuring out how to use it, this has proven to be a big win.

Sauce Labs provides access to automated functional testing. We have been very pleased with that. We make use of the automated piece and the manual piece. There are some teams who like to just poke around with a different browser to see how their app behaves by doing one-off tests.

I have been with the company for three years. I have been supporting it all the time that I have been here. The company had the solution for roughly a year or so before I joined the team.

The stability is great. When I first took over, I was trying to find any general maintenance Windows, etc. so our team could be aware. I remember posting the first few outage notices internally to our customers, saying, "Sauce Lab is going to be down for such and such." Probably because of our lack of adoption, any kind of instability concerns have been really insulated for us.

Latency has not been a concern. Once the pipeline gets set up, latency doesn't make too much of a difference because our development teams are attending their meetings or performing other work while the pipeline runs. Therefore, they are not overly concerned about the speed of what Sauce Labs can provide in terms of latency. At the same time, because the company is so archaic by modern standards with our automated testing footprint, any time savings over somebody going through and clicking the buttons by hand in an application is going to be greatly realized.

So far, scalability has been great for what we need. It is hard to tell its true scalability since we are only running 25 VMs concurrently.

There is definitely room for growth. Across all our separate divisions, people are realizing that automated testing is a major opportunity for improvement. As these individual groups continue to learn from each other and ramp up their own use of automated testing in solutions like Sauce Labs, that will really drive our growth.

We have made use of Sauce Labs’ technical expertise to help us integrate automated testing into our CI/CD pipeline and DevOps toolchain. We have done that to great success with their support teams. I have also had the Sauce Labs team out onsite to the company at one point before the COVID pandemic hit. So, we were able to bring in several people from the company who were interested in seeing how to shape some of it for themselves. Sauce Labs was able to present and provide direct guidance to those teams, and it was a really successful event.

The technical help wasn't overly helpful for me because I am just a really technical guy. I try to figure all that stuff out for myself. However, it was extremely helpful for our team at the company. A lot of the time, most of them don't like to try to figure things out for themselves. They want a dedicated instruction set, rule book, or whatever to say, "Here is how you go and do this." For Sauce Labs to come in and be able to actually show how it was done, I think that provided a lot of value for them.

The technical support team has been great for any kind of issue that my team can't resolve on our own. We were always comfortable opening tickets with the support team. They have always been super responsive and educational when helping us to understand the cause behind the problem, not just how to fix it. I would rate them as nine out of 10 because there have been one or two occasions where it hasn't been absolutely perfect, but it has been really stellar overall.

Sometimes, my customers find a way to go to Sauce Lab support directly, and either they are unable to communicate to Sauce Labs support what their issue is or Sauce Labs is not able to understand them. It is one of the two. Usually, my team plays that middleman and we can facilitate pretty well, providing context to any potential problems or issues within the company as well. For example, if there are specific company systems or things interacting with Sauce Labs that our customer teams may not know about.

On a rare occasion, I will come into a ticket where a customer will have reached out to me after reaching out to Sauce Labs, saying, "Sauce Labs doesn't understand what I am going through. They are not being very helpful." So, I try to do clean up there. Outside of those extremely rare occasions, I have only had one or two of those support issues.

Sauce Labs doesn't know what your test results are. This is a super application specific thing, but it is something that the company struggles with understanding. The app is just the endpoint for the execution of your commands. It can't understand whether the test that you are trying to execute is passing or failing. So, you need to be sure to take that extra step and let Sauce Labs know the results of the test if you want to allow them to assist you in debugging what may have been wrong.

Positive

It provides access to automated functional testing, visual regression testing, and front-end performance, as well as browser/OS combinations, mobile emulators and simulators, and real mobile devices, making it an all-in-one testing suite, which is extremely important to us. This was the primary driver for why we went with Sauce Labs before I joined the company. Having this around is definitely huge.

The solution was adopted before I joined the company.

In general, our test execution speed, time to results, and time to deployment have dropped significantly (by approximately 10%), which is huge.

For the teams who have it set up, Sauce Labs runs with every code commit and provides our developers with immediate feedback. We don't have that as an organization-wide thing right now because we are trying to increase our adoption and execution across the userbase. We are trying to champion this and educate individual teams on how to use it, but they have to develop their own process first. Also, we have regulations in place that prevent us from moving as fast as other companies would like us to.

Going forward, as we continue to adopt not only the function of automated testing, but also the methodologies, best practices, etc., having all their features will be really powerful.

I would rate the solution as nine out of 10. This is mostly due to our company's lack of engagement and being able to fully understand the product and usability for us.

We have been using Sauce Labs to test various browsers and OS combinations as well as test our applications. Our existing automation scripts are written in various technologies, which could be Java, JavaScript, Selenium, Cypress, etc. Jenkins is the tool that we use to typically run our jobs. Through Jenkins, they get scheduled and run in Sauce Labs. This is where we choose to run them, through various browsers and OS combinations.

We use Sauce Labs core and whatever services that we choose to go with, like browsers and mobiles. From there, we can choose the browser and OS versions, etc.

Without Sauce Labs, we wouldn't be able to test as quickly or simultaneously on various browsers. That is the capability Sauce Labs has brought to us.

The automated functional testing, visual regression testing are pretty important to us, as those are the use cases for which we use Sauce Labs. Specifically because it is hosted in the cloud, we don't have to use as much capacity from our own servers. We don't have to maintain and install various versions of browsers nor do we have to maintain them. It is a good solution and works well for us.

Most of our teams who have solutions to test on mobiles are using the solution's mobile emulators and simulators. Having that availability is very important for a complete testing experience.

We use the solution's emulators, simulators, and different browser version combinations for all our testing. For every release, we validate across various browsers. If it is a mobile application, then we need to validate across various Android and iOS devices as well as the previous few versions of each of the operating systems.

The customer base that we typically serve is vast, diverse, and varied. This makes it very common to have our applications used across various applications, systems, and screen sizes. To test across all that, we needed a system in place. That is why we are using Sauce Labs. Without it, we would either develop something in-house or rely on possibly testing in only one particular system. For example, with Android, our entire customer base who is not Android wouldn't be able to validate and reproduce the user experience.

When we are doing releases, the releases happen across all teams and various services. This means all of them need to test for a particular release deployment in whatever environment simultaneously. Based on the number of concurrent VMs that Sauce Labs provides, we can add or reduce VMs as needed. The amount of concurrency that we have purchased is specifically to support our various teams to do deployments simultaneously. Doing these simultaneous tests has reduced our test execution time.

There are a lot of analytics that you can do and look at when you run your jobs. It also gives you how much throttling has been seen across various teams over the past month. It tells you which particular line of code has been failing for however many runs that you have done. 

The concurrent usage and VM availability are its strengths. We have found concurrent throttling very useful. So, we know exactly which job or team is using more than their capacity and might need additional capacity.

We have also found the code analysis that gets run on Sauce Labs very useful. It tells us the line of code that has been failing or not received a new command.

Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. Most of our teams are creating CI/CD pipelines. Some of our teams do many deployments in a day. Without CI/CD, that pace is not possible. Having that capability, for whichever automation tool that we go with, is critically important for us.

Some of the trends that it shows. It only allows you to view the last month. Having it go beyond a month, e.g., yearly trends, would be good. While the yearly trends are available, they are available to Sauce Labs administrators, who are internal to Sauce Labs. I don't know if they are available for customers yet.

My teams use it. I have about eight or nine teams and most of them have been using it at least for a year. Since I joined the company a year ago, I have been watching it being used.

The stability is good. I haven't seen any issues with it. We haven't experienced any latency or downtime issues.

It has been pretty scalable. We are looking at increasing some of its concurrency capacity.

We are currently using about 40 to 50 current VMS.

One part-time resource is primarily needed to design the user groups where jobs will be run. Initially, we had all of our jobs run under a single user name, but then we decided to segregate that and divide it up amongst the teams. So, one part-time resource is needed just to analyze and manage how your jobs get run, then analyze the trends after that.

The technical support has been pretty good. We haven't seen a lot of technical issues in which to engage their technical resources. Occasionally, we used them when we couldn't see something, didn't have an admin account, or for account creation. All those things got resolved very quickly, maybe in a day or so. I would say the technical support is good and rate it as 10 out of 10.

Positive

It was implemented before I joined the company.

It was already set up when I joined the company.

From an infrastructure support perspective, the number of VMs, browsers installations and versions that we would be maintaining without Sauce Labs would be a lot. This includes not only the infrastructure costs, but also the maintenance costs and people's time. The labor cost associated with maintaining all of that would be considerably high. In terms of efficiency, having concurrent VMs with various browser combinations available has allowed us to run multiple executions by all our teams.

Typically, in a night, eight or nine teams have 40 to 50 jobs running simultaneously, which is amazing. Otherwise, if that was not concurrent, then it would have to be maintained in our own infrastructure. That would be a lot of money.

Sauce Labs has room for improvement with its price point. Using a real mobile device, and having that dedicated to your team, costs more than actually purchasing a mobile device. We haven't tried the real devices yet. This is because of their price point.

The number of concurrent VMs that Sauce Labs provides depends on your purchase license level.

Latency has not been a concern due to Sauce Labs being a cloud-based solution. This comes back to the number of VMs and licenses that you have purchased. For example, if I have a capacity of 70 VMs, but I am running 100 jobs, then 30 of them will be throttled and we will see the latency. However, if I were to up my licensing, then I wouldn't see that latency. That is why it just depends on the license tier that you have ordered.

I have not compared it with other solutions.

Plan for how you want to use it and how many teams will be using it as well as the types of accounts that it makes sense to have, different access levels, and who should have it on their team. If you plan ahead, then you don't have to fix it afterwards.

We haven't tried the front-end performance testing.

I would rate it as eight out of 10.

We have two kinds of applications using Sauce Labs in our company. One is the website, and we're using it to test across browsers, such as Chrome, Safari, Firefox, Edge, and mobile, to see that all the components we have in a web page work. 

The second product we use the solution for is our driver application. We make and deliver pizza. Every time an order is received, we dispatch a driver to bring it to the customer. We have an application for driver dispatch on iOS and Android, and we run test cases for those as well.

I cannot imagine our company without Sauce Labs. We are using it a lot. It gives us a provider in the cloud for running test cases, scenarios, and validation, and that is really important nowadays. You could always run testing locally or have your own environment, but it's difficult to scale and difficult to maintain. You need to dedicate a lot of resources to do that.

We ran, for example, 500 test cases in the last week for one specific team, and those test cases were running at 2:00 AM or 3:00 AM, for some specific scenarios. We could not do that without Sauce Labs. We are integrated for continuous integration using Jenkins and we're triggering all these tests nightly for different teams on different schedules. It would definitely not be possible to run that manually or in a different way. The cloud is the right environment to run it in, and for that, Sauce Labs is the best option in the market.

Also, the Failure Analysis helps us prioritize remediation. If we have a very high number of test cases failing for a given feature, we can analyze them, see the root cause, see the code in our repository, and create a Jira ticket to address those failures. That feature is very helpful for us. It's pretty straightforward and doesn't consume too much time. I generally spend about 15 minutes analyzing all the failures across the teams to see which three test cases are failing most.

Another benefit is that Sauce Labs saves us on manpower. We have manual testers, but it's not possible for them to cover all the different scenarios and all the different features before release. That means we need to rely on the automated test cases, and to do that, we need to run those test cases on a cloud platform. Sauce Labs saves us a lot of time doing those validations. It probably saves us from having to hire hundreds of people. Every team within our organization has one or two manual testers, but for every team we have hundreds of automated test cases that we run before release. It's good enough, for our operations, to have one or two manual testers. We are seeing good results with that, but to run hundreds of different scenarios I don't know how many people we would have to hire.

From my perspective, as an automation architect, the most helpful feature is the test history. I can easily go to the dashboard and see how many test cases we ran and how many failures and errors there were. I can segregate things by team and by specification. And I can tell a teammate, "You need to fix this test case, it is failing too much. The percentage of failure is too high." 

The most important historical data is for the last seven days. I don't go too much beyond that period of time because my feedback is about how we did in the previous week. In our environment, everything is changing all the time. We are testing different products and running different test cases. So for me, it's the recent data that is key.

The Failure Analysis feature is really important for us, one of the most important aspects. What is the root cause? Is it because we have a defect or is it that we have a test case that we need to fix or modify? The Failure Analysis is one of the main functionalities that I am exploring all the time in Sauce Labs. I can see a test case that is failing and on which particular platform or device it is failing. And the most important part is identifying the problem with the code. We can always go to the line of code and see the possible solution. The Failure Analysis helps us to discover which test cases we need to work on.

Before the test history became the most important feature, the most important part was the configuration that made it easy to connect test cases to the cloud. As a result, we can easily test different devices at once. 

And from a manual test perspective, testing in different browsers using the live test solution in Sauce Labs is very helpful.

Every time that we run scenarios where we need to discover the geolocation of our customers, by default it shows as Palo Alto, California. That's a problem for us and we need a workaround for those cases. It would be great if Sauce Labs didn't provide any geolocation by default, and we could provide the geolocation that we want. It would be helpful if we could enter a latitude and longitude into Sauce Labs configuration and say, "When you run a virtual Chrome device or an iPhone, make this your default location. Then, provide me that device so I can run my scenarios," because we have stores in different regions across the United States.

That is the only "ask" that I have for a feature, a geolocation that we can set by configuration. That would be awesome.

I have been using Sauce Labs since I started with this company, back in 2018, so that would be about three years and six months.

I am an automation architect and my work includes providing cloud testing to different teams. Sauce Labs is one important piece for us.

It's pretty stable. We don't have any problems with the solution. We probably need more virtual machines internally, but that's not related to the performance or availability of Sauce Labs. When it comes to stability, everything is going great.

The scalability is connected to our budget. The virtual machine concurrencies that we have are not great for our demand. I am talking with management about increasing the budget for 10 or 20 more virtual machines. But Sauce Labs itself provides the opportunity to scale very easily. For us, it's just a matter of budget.

Sauce Labs has a pretty reasonable price and is worth it for the solution and what we are doing with it.

There are a lot of other options on the market and I have tried some of them in the past. But Sauce Labs has a lot of advantages. For example, one feature that I really love is that when we are doing troubleshooting or have a problem with test cases, they answer every single question so quickly and are very helpful. The customer experience using Sauce Labs is great in terms of support. That is an aspect other companies don't really have.

I started with our company as an automation engineer and their support was able to help me with every single configuration, every single problem, and every single question. I cannot remember a single time that they were unable to help me. They are the best.

The usage of Sauce Labs was not as great, when I first started using it, as it is now. We have a great approach to testing using Sauce Labs. We test mobile and we do live testing and we are running automated testing. We're integrated with CI. Nowadays, it's better than three years ago and I look for it to improve even more in the next year.

My company uses Sauce Labs to run all smoke and regression tests for our application.

We primarily run our tests on Chrome 83, but we occasionally run on other versions (81 or 84) to check to compatibility. Our smoke suite runs on a nightly build, and regression on a bi-weekly basis.

For building out new automation features, we have to have a passing Sauce Lab run before creating a pull request. Doing this ensures that our new automation features will run on other environments and not just locally.

The insights section provides a great overall state of the automation suite and can identify trends relatively quickly. If we see a dip in our passing rate over time, we can look at what changed when the test started failing to find the root cause rather than doing a quick fix to find that the test fails a short time later.

The visual data is helpful for all levels within the organization from the QE engineer level up to the director level.

The ability to change the browser version has also ensured that our application stays compatible with previous and updated browser versions.

One major feature that I like about Sauce Labs is the recording feature. I love the option to watch a video playback and command and view log when I'm trying to figure out why my test was passing locally but may fail on Sauce Labs. It's helped to make sure of my automation.

Another feature I find valuable is the ability to choose browser versions. This feature allows us to test on our minimum browser version but also check/verify that our application is working on the most recent browser version as well.

The insights tab is also very valuable, as I’ve discussed early; it provides a great deal of data and determines trends in our automation sooner.

From a company use standpoint - I have no complaints. The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended. Personally speaking - I like to maintain my automation framework and would like to occasionally test that it continues to run on Sauce Labs after releases without stealing from Sauce Labs.

I've been using Sauce Labs consistently for a year now since I've started with my company.  After being introduced to Sauce Labs, I've incorporated it into my framework that I maintain.

A previous solution I had used was running a VM from a NAS server to run our automation scripts.

Be sure to plan out your solution before starting the free trial; 14-days goes by rather quickly.

Sauce Labs was already integrated into our framework and SDLC process when I started with the company.  After using it for a year now, I would highly recommend this to anyone.

We use it for a lot of end-to-end UI test automation.

We really just use the visual test automation, not the performance, for our product teams.

There are a couple of products using this solution, so it has been nice having a visual result. We can step through the tests, if necessary, and showcasing to the business that we are doing automation has been very helpful.

I like the dashboard and seeing the test results. As a manager, I like to see the insights of the people using it, understanding the total path and run. I can see all of that as a manager. I also know team members love seeing the dashboard and seeing the test results in real-time.

Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. That is just what they do. It is an easy, one-click solution versus having to manually run scripts.

The platform is easy to use. I can quickly navigate it.

Sauce Labs provide access to automated functional testing and visual regression testing from a single platform. It is very important to have one tool that can do this versus having multiple. Just integrating Sauce Labs within the application and seeing the pass-fail results right away is extremely helpful for teams. You don't have to go to some place else to find them.

Everything is so intuitive.

I can't remove team members that have left the organization. I can only set them as inactive. It would be really nice to clean up my data and delete them from the team management.

I have been using it for several years.

I haven't heard anything about stability. It has been pretty fantastic.

Latency has never been a concern with Sauce Labs. Being in the cloud is super fast for any application, and we take advantage of Sauce Labs being in the cloud. If there was any latency, then it would be on the coding of Selenium versus the Sauce Labs application.

We have about 60 products and only three or four are using it. We have started getting other team members' products to use and integrate Sauce Labs in their test automation, though I am 100% sure of what every single product is doing.

The support is fantastic. We have monthly meetings with Sauce Labs. Even if I have questions, I can just reach out to them. If I have any questions, then all I have to do is reach out to them for help, and they are very responsive. If I have any questions on training, they are willing to set up training calls and train the team on Sauce Labs.

The feedback is super fast. They are always there when I need help. I have never had an issue with their technical support.

I would rate their technical support as 10 out of 10. They are constantly reaching out to see if we have any problems or need anything, but we are perfectly integrated with the Sauce Labs application and don't have any issues.

If Sauce Labs doesn't suggest training, make sure that you ask for it, if needed. They are always willing to come out or even do virtual training. Whatever they can do to help with the process, they are always there. So, if you don't hear about the training, make sure you ask.

Positive

We did not use a different testing tool before Sauce Labs.

The initial setup was very straightforward. They make it so easy to be able to integrate Sauce Labs with Selenium. There were no issues at all, and if we ever had issues, the technical support would just help right away.

We partnered with Sauce Labs. They came to our office for a few days and gave a demo to a large group of people on how to use Sauce Labs, but also provided a little training on Selenium.

I liked the training. As a manager, some people hesitate using test automation because they just don't know how to start. By having some hands-on training with Sauce Labs team members, we were able to give that technical training to people so they understood, "Oh, I could do this. I can start this."

They saw how easy it was to integrate Sauce Labs within Selenium code. The training just helps make it possible because I can't make people do things, when it comes to test automation. By being able to train hands-on and seeing the results, that made it possible for us to do test automation.

We have definitely seen ROI. We are utilizing Sauce Labs specifically for one of our product teams with several more being integrated with Sauce Labs. Seeing the benefit from even one team continuously using it, we are happy with the results that they are getting and having them sharing that knowledge. That is really how we will get other products on it, when people see that it is working for teams who are getting positive results. 

I am not in charge of the licensing costs. They come and ask me, "Do we still need this?" Then, I tell the teams why we need XYZ and the licenses.

We did initially go with Sauce Labs because of the pricing and integration.

I didn't make the decision on Sauce Labs. It was recommended by some other team members that did the research. People were looking at, "Can it help us from a visual point of view? Can it add tests, see all the tests through portals in the cloud, and make testing go quickly?" It offered the possibility of being able to run on multiple browsers and operating systems as well as had integration with Selenium.

I have no complaints nor issues with Sauce Labs at the moment. It meets all of our expectations.

Teams are continually using the product. I would hear complaints if there were any issues. I am trying to get teams to spread the word to other teams. So, the teams are very happy with it and trying to get other people to use Sauce Labs as well, then integrate that into their product team.

Because our applications are not mobile, there are only certain browsers that we need to support. So, we are really not taking advantage of the number of browser/OS combinations, mobile emulators and simulators, and real mobile devices that it offers. I am just trying to get team members up and running with test automation and having some tools to be able to help them with that. I know that they are very competitive with other products in this regard, but that has not been an important factor for us.

I would rate the solution as 10 out of 10. I love the capabilities of it, the support, and the service. 

We mainly use it to run our test cases for different platforms. We're able to run it for multiple browsers and multiple devices. We use it for about 90 percent of our CI/CD test cases.

We are using it for automated testing and not for visual testing or performance testing.

It's important that the solution gives us access to automated functional testing, as well as browser/OS combinations, mobile emulators and simulators, and real mobile devices, all in a single testing suite. That's particularly so for our client-facing applications so that our customers are able to access our products and offerings through any device and platform. For those types of applications, it is a huge help. The number of browser OS combinations, mobile emulators and simulators, and real mobile devices it offers is really good. There has not been a case where we say we want a given combination and we don't have it.

One of the most valuable features is that we do not have to have the cross-platform testing vehicles in-house. Sauce Labs gives us the ability to test across platforms and that really helps give us confidence in our products.

One of the challenging areas for us is the reporting and the matrix. It should be based on roles, but right now it is only available for the admin role. The admin role can really do a lot of stuff, but our infrastructure team, which holds the admin role, is not ready to hand it over to us on the testing team. If Sauce Labs had permissions associated with roles, where this role could do this activity and that role could do that activity, it would be easier for us.

We have raised this suggestion with our product manager and she has put it in the product backlog, but, of course, she cannot guarantee when that will be looked at.

I joined this company seven months ago, so I've been using Sauce Labs for that long. But the company has had it for about three years.

From a solution stability point of view, it's good.

We are able to run some of our tests in parallel on the solution, within the constraints of how many licenses we have. We have some 50 testing teams across our organization, and we use them. There has never been an issue with not being able to execute tests.

Latency, due to Sauce Labs being a cloud-based solution has not been a concern.

We did have downtime in late September last year, but it was because of our lack of knowledge in terms of the configuration. Sauce Labs support was really quick in responding and triaging and fixing it, which was really great.

With scalability, they are going in the right direction. I have attended some of their new product webinars on API testing, and I really found that to be cool. We may use it, or evaluate it at least, once we start going into that direction. That goes for visual testing as well. If they are able to deliver on some of their promises I really look forward to that and seeing how we can utilize them.

We could use Sauce Labs even more. While we are using it in our testing teams we are not yet there for things like API testing and visual regression.

Our next step, in terms of our exploration of the solution and how we want to use it as part of the CI/CD, is that we are moving into GitHub Actions. We were using Jenkins and it worked well with that. We are moving to GitHub Actions and trying to figure out how that will work.

Overall, technical support has been really good. They have responded to our questions and have triaged pretty quickly and followed up.

The reports that our account manager at Sauce Labs provides are helpful to us to see where our gaps are. She also helps us evaluate where we could be from an industry-standard perspective. She gives us an idea of what others are doing, what she's seeing out there, and in which areas we can improve.

In terms of using Sauce Labs’ technical expertise to help integrate automated testing into the CI/CD pipeline and DevOps toolchain, we have not done that yet. That is more due to the way our organization is structured. We, as the testing team, do not have access to how the tool is managed or how it was set up. There are probably gaps in terms of our understanding of how the tool can be used effectively. But we have discussed this with Mandy, our account manager, and she has offered to do a free session for our associates, which is scheduled for next Friday, and we are looking forward to that. It should help us understand what the best practices are and whether we are using it right.

From the perspective of our organization's roadmap, we are new to testing automation in general. Our focus is on automated functional testing. There are certain use cases for visual testing, and there have been talks about performance testing, and an evaluation will be going on for that. As part of the workshop next week, Sauce Labs will also be presenting some of their offerings, so we'll be looking at those as well.

Positive

Sauce Labs is a tool. It can fit the needs that you have, but you first have to figure out your needs. Once your team has been able to successfully use it, and things have stabilized, go to the next need you have. That's exactly what we are doing. Once we get our functional testing to be stable, we'll look into the next performance test case.

Don't try to do everything together. When you're going with a new solution, it takes time for people to adopt it.

The process could be different for a large company like ours, versus a smaller company where there are just a couple of products. They may be able to move faster. For a financial institution like us, there are so many roadblocks that we have to go through. I do understand that the licenses are expensive, and you do not want to get these licenses and have them sitting idle.

I would rate Sauce Labs a 10 out of 10 because I have not been able to find any use case that Sauce Labs does not provide.

They are our preferred vendor for all our mobile and browsing test needs.

We have been using it for mobile applications and multi browser testing and multi-device testing.

I use the mobile application. Whenever we get mobile builds through the CI/CD pipeline, we directly upload them to Sauce Labs. We usually test on various devices and versions. The solution helps us to identify any issues that we might face in different environments, devices, and versions.

We can run our tests in parallel. We have access to all the browsers and devices, where we can run our test haphazardly. They have video recordings as well for the tests that we run, which can easily be accessible and shared across different teams or management without them being actually logged in. 

It provides a comprehensive selection of browsers and platform versions for our test automations and CI/CD pipeline process. It also provides a comprehensive set of virtual mobile devices, which we can configure for our automation and availability. These features are valuable for us when it comes to testing our applications. We have a website and mobile applications that we want to test and diversify to various browsers and mobile devices as well as restore various versions. This helps us to find bugs that users might be facing and correct them.

The real concern is the load time of applications or real devices when we start our tests. It takes some time to load the application or web browser. Sometimes, it is frustrating too. Since they are real devices, we understand it takes some time to load. However, if it were to improve, then that would be a great asset to the solution. So, we would like better responsive times when opening applications and running tests.

We have been using Sauce Labs for more than two years.

It is stable. We haven't explored all their platforms yet, but as far as we are concerned, we have seen stability.

It is definitely scalable.

Mostly, our team consists of 15 people who run tests automation and test manually. We have other teams as well who have access to Sauce Labs, and they test certain applications in Sauce Labs. We guide them on how to use it. So, there are around 25 to 30 people who probably use it quite often.

We use it to do at least 10 test automation runs every day. We do have plans to increase our usage in the future.

I worked with their customer support for these devices. For the issues that we were facing, they were very responsive. 

Other than the automation part and CI/CD process, one of our colleagues works with them. I mostly work on the manual efforts of Sauce Labs and our applications while one of our teammates works with the automation support.

I would rate them as nine (out of 10). I would even give them 10 at times. They were very responsive and quick. When we were facing some issues, they usually responded quickly.

Positive

Before joining my company, I used to do tests manually.

I was not involved in the initial setup, but I was able to quickly adapt to using Sauce Labs after joining the company.

Running tests in parallel on Sauce Labs definitely saves time and effort. Less time is consumed and we can have various results on our various versions or devices. This ultimately impacts our business by identifying the real threat and real defects, then moving forward accordingly. 

Usually, our test runs for about 40 minutes with various levels of testing. Sauce Labs reduces that time in half.

We have seen ROI. With some improvements, we will be much happier with our ROI.

The pricing is reasonable due to the amount of diversity that they provide. However, I feel they might be more flexible to bargain based on their relationship with our organization.

They can increase the range of real devices for testing while reducing the cost. When it comes to purchasing real devices, even if we have a certain number of devices available, it would be great if we could add additional real devices for a reduced cost. 

I would like to give the admin the ability to upgrade operating systems based on the client's needs rather than having them update the OS versions every time. Maybe they could create a page for admins where all devices or browsers can be purchased or viewed, so we could then have an idea of what kinds of devices could be purchased.

I would rate the solution as eight or nine out of 10 based on the diversity that they provide.