Broadcom Service Virtualization Reviews

It gives us the ability to simulate and virtualize components that we don’t currently have. This is invaluable.

At the moment, we are still rolling out elements of the product. But we can already see that it gives us capabilities we haven’t had before, and we are very happy with it.

We had a complex initial setup in our environment, but CA helped us with it.

We have no issues deploying it.

So far, it has stood up really well; no issues at all with stability.

We have not gone down the scalability path yet, but hopefully next year we will when it becomes relevant.

The technical support team is very smart. We had a CA architect come to us to help jump start the operation. We were pleased because he learned our business and helped us get the solution off the ground.

We previously used a different solution. We switched because there was a gap in our testing which we needed to fill in.

Getting it working in our particular environment was complex.

We looked at HPE, but ultimately we chose CA because they were able to meet our challenges. My biggest point was to prove it in our environment. CA stepped up and did a proof of concept. It took six months and we gave them the biggest challenge and they figured it out.

My advice would be to look at the solution in detail, as it can fill a big gap in your testing. It, of course, does require tech knowledge to work, but for what it delivers, it’s worth it.

We were using this product for service virtualization, and SoapUI/REST API testing for the US’s largest retail customer in the home improvement space.

For service virtualization, we had a lot of subsystems in our application; and we had lot of dependencies on the downstream applications. We extensively used service virtualization to reduce these dependencies, which could be due to readiness, or downstream applications having downtime. We also had few third-party subsystems whose technical support was not available during US night hours. This resulted in a complete loss of the offshore effort in India. This was hampering our release dates, which are very critical in the retail business. Missing a release has a directly proportional impact on revenue.

For web services testing (SoapUI/Rest API), we initially had our web services test cases automated using a custom built Groovy/Selenium/SoapUI framework. Though the framework was robust, it was difficult for us to maintain the automation scripts. When our customer bought CA Service Virtualization, we were able to move our 3000+ automation scripts to Service Virtualization scripts within 6 months while supporting new features and producing regular releases. This was commendable.

Training our folks on Service Virtualization was also pretty easy. We hired three Service Virtualization experts from the market, and were able to build a team of 14 Service Virtualization experts within three to four months. CA Service Virtualization is a very robust product and is highly configurable. This helped us to take more user stories, compared to the days when we were using our home grown framework.

To the best of my knowledge (and memory), listed below are the few challenges we were facing at that time. It could be possible that CA would have already addressed a few of these limitations and short falls.

• Simultaneous execution of automation scripts was not possible; or, at least, not readily available. We had to write some wrappers to handle this situation.
• Integration with Jenkins was not available. We had to write small utilities to achieve this.
• Integration with JIRA was not available to automatically log a defect if there were script failures.
• It consumes more memory and makes the system very slow, which was stopping simultaneous execution.

I have used this solution for more than two years.

As mentioned in the room for improvement section.

I don’t remember any issues as such, apart from the ones mentioned in the room for improvement section.

As I mentioned, we were using a home grown Groovy/Selenium/SoapUI framework. Once a customer bought the CA Service Virtualization license, we were forced to use this. However, we found the product to be more robust than our framework.

Initial setup was pretty straightforward in terms of installation and configuration. Later, as we encountered a few challenges, we built a couple of utilities.

It is pretty expensive. On the licensing part, we don’t have much insight because the customer was handling this part.

We had evaluated multiple frameworks because we were working in a multi-vendor scenario. Every vendor brought their frameworks and solutions to the table. I don’t know if the customer evaluated any other commercial products. A few of our team members had already worked on iTKO/Service Virtualization on other projects; and they provided good feedback on the product.

I would recommend this product for clients who are heavy on web services. This product is very easy to learn, implement, and configure.

For us, the most valuable feature is that it gives us the ability to virtualize a lot of our integrated systems. That’s valuable because we have difficulty trying to coordinate all those different systems in terms of scheduling, timing, refreshing of data, and that type of things. It gives the developers freedom to develop without needing to wait for others; so it eliminates bottlenecks for us.

We were having a lot of bottlenecks in our development process. Developers were waiting around for somebody else to finish and that helped alleviate that.

I don't think we've gotten enough experience with this version to talk about what should be in the next version.

It could be easier to setup and to deploy.

We have just started with the product. We're probably about six months in.

We haven’t noticed any stability issues.

Technical support was responsive. They did a good job.

The initial setup was straightforward. It required training, but it was straightforward after the training. The training helped get us off the ground.

We did not have a solution before. We went with CA Service Virtualization because of their ability to have a long-term customer relationship, and the ability to work with us to get the product off the ground.

Get your business partners on board. Do an ROI analysis where you can actually have concrete use cases to sell the value of the product.

The biggest value for us was writing all the stories and figuring out what stories are going to get into which sprint. It also helps us with organizing the backlog and dependencies. We've found those features to be useful to us.

In terms of improving the way the organization functions, it gives us lot of visibility into what features are getting released when. It also helps us track across the organization in a much better fashion. I think visibility is the biggest benefit that we got out of implementing CA Service Virtualization.

We would like to see more robust dependency management. This is the only thing that comes to my mind.

We have not had any issues with stability. I think things have been working fine for us so far.

We've started a little slow. There are just a couple of teams that are using CA Service Virtualization, so we have not really scaled it up for multiple teams across the organization. But within our group, there are like 3 or 4 teams that are using it right now.

We have not used technical support so far. Thankfully, we didn't have to use them.

We did not have any previous solutions, but our organization was moving towards this. We evaluated a few products and CA seemed to be a good option that will fit all of our needs.

I was not involved in the initial setup, but there was somebody in my team that was involved. I don't think we encountered a lot of issues or challenges as such. We had to, of course, pull some experts from CA as well just to get their ideas on how we're supposed to set up, how things work, and what are best practices. Other than that, the first couple of sprints we had to learn and tweak a little bit as part of our retrospectives. Otherwise it's been pretty much okay.

We had Pivotal Tracker and probably a couple of other vendors as well. I don't remember their names. We went with CA Service Virtualization due to the user interface. We found it to be more intuitive than the others. We felt that it can probably scale up to the needs of what we have within the organization in terms of integrating with the rest of the ecosystem.

Potential users need to clearly know what they want, how they want to implement it, and how they want to use it. Then, once they have clarity, they can figure out how they want to set up the solution and go from there.

The web services protocol for SOAP requests and responses is the most valuable feature. It is easy to set up the test for them without much coding, thus it is easy to train new testers to work on it in a graphical development environment where one can see the flow of information visually like the TIBCO product. It is easier than SoapUI to follow.

This product has introduced a simple-to-use middleware tool, which helped our team conduct web-services testing.

The most important thing is to have integration with Excel. It has a way to read the Excel sheet but not to write onto it. If I need to generate customized reports in LISA in Excel sheets for management purposes (which can show me number of test cases pass/fail/not started, etc.), I can't do this since you can't ask Service Virtualization to put specific values in particular cells with a particular color, etc. We were able to achieve it only using Java, which meant a lot of scripting, error debugging and rework, and this defeats the purpose of it offering a codeless environment to testers. A simple groovy scripting in SoapUI can achieve all of the above.

The Service Virtualization section needs to be more intelligent in offering a good solution to customers. For example, the virtual service created via the request response pairs needs spoon feeding of responses before the service starts performing. And I am not sure if it serves the purpose of virtual environment till the actual system becomes available. I have yet to see a proper implementation and successful usage of a virtual service through it.

CA should provide a free trial version of the tool (was not available till version 7.0.10) so that clients can try it out before making a purchase.

I've used it for two years.

There were no issues with the deployment.

The product is yet to become stable. I have seen instances when the tests have not been repeatable in LISA when scripting is done in Java. After removing and adding a character to the code, the test starts working again. Sometimes restarting it gets it working. Also, there are caching issues in LISA and it picks up the old values of the variables/parameters/properties in the tests even though the values have been updated and saved in the test before commencing the run. Thus, people prefer using more stable solutions like JMeter and SoapUI.

There were no issues with the scalability.

The customer support for LISA is average with the associates not being reachable all the time. Also, there were no specific communication protocols set up to interact with LISA support team and we were mostly relying on the Confluence forum to have our queries resolved. Also, the search results on Google do not provide answers or hints to the questions LISA testers face, unlike SoapUI and Ready API which have a lot of online support available. One can only rely on the user guide of LISA which does not cover all the scenarios.

The initial setup was straightforward, but the settings procedure, like providing the correct domain name, server name, username and password, did not work the first time they were provided from LISA.

I got it implemented through a vendor team. There are a couple of settings you have to play around with before it starts working.

Choosing this product was a management decision and we would prefer other solutions because in order to achieve a customized solution one would have to do scripting. This tool needs to offer a lot more options before it can be marketed as a codeless environment that can really help an organization chive results with such a testing team that is not good at coding.

QTP can achieve almost everything to do with scripting testing. Other tools like SoapUI/Jmeter being more popular become the obvious choice.

It's got a standardized format for endpoints as well as containerized, virtualized endpoints that are used by everyone on our team. We can spin up these endpoints if necessary.

We're still working with it, but it's given us a common deliverable. The overall testing strategy allows us to have more common strategies amongst various products.

The user interface and usability of the tool is not the best. They could definitely be improved upon in the next release.

Also, we are hamstrung by different releases and are not using all of the features, but it is useful.

It’s not stable as of yet, as it needs a lot of infrastructure.

We haven't done so yet, but we're hoping it will be scalable.

We interact with an internal department that works between CA and ourselves. They have all of the info we need.

I think we were just using many different tools to achieve the same thing. We needed one tool to do it all, and this fit the bill.

The initial setup was straightforward.

You should make an overall testing plan, otherwise it’s difficult to implement quickly.

I’d compare it to open-source solutions that already exists. Consider your organization before diving deep. For us, we are building up our maturity, and this solution is built to a higher standard.

We do testing, so obviously the virtualized service is going to be our big use. A lot of the times, services and/or components are not available, whether it's because the equipment is not available or is being used for something else. With Service Virtualization, we can emulate services to test against them instead of waiting on infrastructure.

And it does it pretty well. We do the recordings and get the response-request pairs. We also have development groups that are probably going to be using these services because they won't let us test earlier in the life cycle.

We're actually ahead of the development groups on this, and they're looking into docking it, but they don't know how use it and they don't know how to create their own services. So we'll probably create the virtualized services for them and we'll support it, and then the development groups will test against our stuff.

We're fairly new adopters as we've only been doing this for less than a year. Because we didn't know what we were doing in the beginning, there was probably two months of ramp-up when we had to learn, and now we're getting more and more comfortable and we're getting pretty good at it.

The features that I think we need weren't in 7.5.2 and the 8s. They're all going to be in version 9. These are some reporting and scalability features, as well an expanded virtual service in version 9.

The automated generation of tests could be improved. Right now, we have to generate them all ourselves. We want to be able to run that against the service and have it creates our tests.

It really isn't scalable, other than the licensing. We can pretty much test away and we can easily clone services if one group needs a slightly different variation.

Technical support has been great. They've been supporting us well and had people on site, which helped a lot with our architecture questions. We had a three-day class, but that's not the same as really them helping us out.

No other vendors were looked at.

It's better than I would've originally anticipated. Just understand there is a slight ramp-up, but once you get past that, the value is really there. I think, wow, we can do a lot of stuff, save a lot of time, and save a lot of money.

When we previously used CA SV we saw tremendous results, and went live with almost a 0% defect rate.

We use it for Service Visualization and to run tests to validate our services response with the database.

With CA SV it took 2.10 minutes as opposed to 210 minutes for a total validation per scenario. This reduced the development cycle time and provided a product with no defects from a QA perspective.

Performance and cost.

I used it a year ago and recently started using it again.

CA SV takes up lot of your system resources.