Broadcom Service Virtualization Reviews

We are into performance engineering. There are a lot of back-end systems that are sometimes unavailable, or there are a lot of various dependencies, and we're trying to get rid of those. That's one of the reasons we got into DevTest. 

And it's been helping us quite a bit now. We've been able to step out a lot of back-end systems. It's performed pretty well. There were a few hiccups, but we got in touch with the senior architects and they helped us out with fine tuning and, since then, it's been working out well.

The most valuable feature is that it supports so many protocols. We, being a large bank, have almost all the protocols, and it supports all of them, so that's one good thing.

I think it definitely helps us reduce time to market, and in terms of dollar savings it helps us quite a bit. There are many testing applications that reach out to third-party vendors where every transaction costs, and we are able to virtualize all of that and reduce the financial aspect.

One major feature I would like to see is on the user administration part. Right now, anybody can access any of the folders and any of the projects. If they could bring in someone amount of user administration, that would help us a lot.

Stability so far has been good. There have been no issues. There was a learning curve, about a year or so. It took us some time. It's not a plug and play tool. There was a lot of customization that had to be done, but with all of that done I think it is quite stable now.

The hiccups we experienced were mostly to do with scalability.

There are multiple protocols that we actually work on, like HTTP, HTTPS and then MQ and TCP. HTTP and HTTPS were fairly basic, and we were able to get things done pretty efficiently. When it came to MQ and TCP, we came across a lot of challenges. That's when we got CA engaged and they helped us fine tune the tool itself so that we could reach the scale that we wanted to.

I would say they're a four out of five. A lot of times what happens is, they come back with scripted answers and then we reach out to the account manager or somebody. Then they look into it and they help us out. That's why I'm taking out one star for them. Otherwise, they've always been good.

We were on the lookout for a service virtualization tool. 

I would say it was complex.

The initial set up, if you look at the basic functions, it's quite straightforward. But for the scale that we were planning to implement it at, we did not get a lot of support from CA. So a lot of it had to be done in-house. We had to bring in our own architects and do a lot of environment setup and the like. We did not get a lot of support from CA. That's one area where I would say CA could have helped out better.

We did look at a couple of tools in the market. We did a PoC on DevTest. Back then it was called Lisa. Everything went well, so that's how we ended up getting into DevTest.

We didn't do a POC on the IBM product, I think it's called Green Hat - IBM's has a service virtualization tool - and we were actually contemplating that tool as well.

We went with CA because the PoC went well. There were a couple of guys who came in and actually helped us out with all the use case scenarios that we had, and we were able to implement it successfully. That was one of the reasons we went ahead with CA.

When selecting a vendor, our criteria, what we are looking for are 

• easy to use tools
• a good feature set
• and then awesome support.

I would say it's an eight out of 10. Eight because it has so much going on and there's a lot to learn, and there are a lot of things you can do with the tool. It requires quite a lot of learning effort. It's not plug and play. And a lot of customization has to go into it.

I would advise going with DevTest, definitely. It would depend on what you are looking to get out of the tool. If it's some of the basic protocols like HTTP, HTTPS, you could actually do it yourself. But if you're looking at a much larger scale, then definitely DevTest is the right tool to go ahead with.

Virtualization is very important for me so as to mitigate my dependency on real back-end system and push the QA testing life cycle earlier.

Especially in QA/Test/Pre prod environment because of rapid changes and non availability, CA LISA is helpful in replacing the back-end with stub responses which can be helpful in pushing the QA life cycle much earlier in the overall development cycle.

It's currently very sensitive to app changes/responses & the setup to capture the responses is a bit cumbersome .

3 years

Only some security issues with SSL/RSA.

Setting up the proxy to capture responses for the model.

We didn't encounter any issues with stability.

Right now we're only using it for pre-production so we haven't yet seen how it scales if we use it for production volumes.

Pretty Good.

Good

Ease of use and reliable solution.

It's a bit complex as far as creating the model and then improvising it. Also maintaining the model is cumbersome.

We implemented via a vendor team - their support was good and they had a high level of technical expertise.

ROI is NOT that great since the product license is expensive.

Pricing and Licensing is expensive.

Over the past 3 years which I've been using it, I've found it to be a good solution. Only drawback is price, setup & maintenance.

I like the fact that it allows us to focus on just the system under test. We can remove of all the dependent requirements from the system.

It allows us to implement an Agile Shift Left development paradigm so that we can start testing things before all the pieces of are completed.

I think the biggest improvement can come in usability. It's still very complex to use. Also, we've found a fair number of bugs, but CA always resolves them.

I would say it's been mostly stable. We did have a problem related to a service one of our teams was running that was not working properly. But, we don’t yet know what the root cause was.

It's scaled to meet all of our needs so far.

Technical support has been good. They're responsive to our requests.

All our teams were using their own solutions. We wanted something more universal.

Initial setup was straight forward. It was an easy install.

We also looked at Parasoft, but we thought that CA met our vendor requirements more closely.

When choosing a vendor, the most important criteria is whether we can have a long-term partnership with them. Obviously, cost is also an important factor, as well as software reliability. We also look at the future road map for the product.

It's important to have a group that really learns and understands service virtualization well. You can then leverage them for any teams that want to implement SV.

It provides for automated testing of applications, including the following features:

• Wide technology support (SOAP/REST web services, message queues, web UI using Selenium, database queries);
• Support to build modular test cases using reusable blocks (called sub-processes in DevTest);
• Extensibility using Java;
• Integration with Jenkins continuous integration tool; and

• Detailed test execution reports.

We've already created and switched over 10,000 test cases from manual to automated execution. Our QA team can now focus more on complex manual test cases.

I would like to see a completely web-oriented automated test case design (currently need to use desktop application called DevTest Workstation). I'd also like to see IntelliSense-like functionality for custom scripting, as well as improvements to reports for test cases using datasets to quickly point to/locate results for specific data iteration.

I've used it for one year.

We experienced some minor issues, but they've been fixed in the current release.

We experienced some minor issues, but they've been fixed in the current release.

We experienced some minor issues, but they've been fixed in the current release.

Very good, as support person is usually trying to do even more than expected. There’s also a web site by CA called Communities where product users can help each other.

I’ve used HP QTP/UFT tool for other clients, specifically for automated GUI-based testing of desktop applications (.NET/WPF, Stingray, Java Swing). I can’t directly answer question about choosing CA DevTest as I was not in the tool decision group for current clients, but it’s very suitable for SOAP, REST, JMS, DB testing. CA DevTest would not be suitable for GUI-based testing of desktop applications which our current client does not need.

The initial set-up was not complicated thanks to the detailed documentation available.

We used an in-house team with occasional help from the vendor support team.

I believe this product will meet your expectations if you’re looking for automated testing solution for any of the supported technologies. It also offers other major functionalities such as CAI for transaction recording and automated generation of test cases and virtual services which we’re planning to use.

We do testing, so obviously the virtualized service is going to be our big use. A lot of the times, services and/or components are not available, whether it's because the equipment is not available or is being used for something else. With Service Virtualization, we can emulate services to test against them instead of waiting on infrastructure.

And it does it pretty well. We do the recordings and get the response-request pairs. We also have development groups that are probably going to be using these services because they won't let us test earlier in the life cycle.

We're actually ahead of the development groups on this, and they're looking into docking it, but they don't know how use it and they don't know how to create their own services. So we'll probably create the virtualized services for them and we'll support it, and then the development groups will test against our stuff.

We're fairly new adopters as we've only been doing this for less than a year. Because we didn't know what we were doing in the beginning, there was probably two months of ramp-up when we had to learn, and now we're getting more and more comfortable and we're getting pretty good at it.

The features that I think we need weren't in 7.5.2 and the 8s. They're all going to be in version 9. These are some reporting and scalability features, as well an expanded virtual service in version 9.

The automated generation of tests could be improved. Right now, we have to generate them all ourselves. We want to be able to run that against the service and have it creates our tests.

It really isn't scalable, other than the licensing. We can pretty much test away and we can easily clone services if one group needs a slightly different variation.

Technical support has been great. They've been supporting us well and had people on site, which helped a lot with our architecture questions. We had a three-day class, but that's not the same as really them helping us out.

No other vendors were looked at.

It's better than I would've originally anticipated. Just understand there is a slight ramp-up, but once you get past that, the value is really there. I think, wow, we can do a lot of stuff, save a lot of time, and save a lot of money.

DevTest supports multiple testing environments so you can configure it to use one testing environment or 20, and you can change it for use in any other environments with the flick of a button.

It takes a bunch of different technologies and you can use it all in one solution, so I can work with my databases, my REST service calls, different web services; and I can use it all in one place. We do a lot of integration testing, and that takes care of it all.

I think the one thing that we need that it doesn't do is performance testing for UI applications. So it integrates with Selenium, but we can't use it for performance testing.

We'd like to be able to do performance testing for in-house applications with more than five to 10 users, but we can't do that with the tool.

There's a couple of things with the UI that make it a little bit hard for new people to use. It's not intuitive in some places. You have to know what you're doing. It's not something that somebody could pick up and just start using the same day without somebody explaining it a little bit.

It's been very stable. We haven't had any issues with it.

We're not using it. We are using it in a QA group, and so it's a pretty small adoption right now. We're trying to increase that.

Initial setup was pretty straightforward.

There was another one called Green Hat by IBM. We were looking at that. DevTest seemed to be a lot easier to use. Also, it worked with TIBCO and with other technologies that we were using.

Definitely read the documentation. The documentation is pretty good.

Record and learn response patterns: It helps us to record traffic and learn automatically response patterns. This feature helps us to virtualize quickly and build an exact copy of the web services.

This product helps us to:

• Simulate broken/nonexistent systems
• Avoid latency between project
• Perform load bearing testing on unavailable external systems

More docs/code samples to develop custom steps: Sometimes we need to develop in web services custom behaviors that simulate technically and functionally the real service.

I have used it for two years.

Technical support is 9/10.

Service virtualization is the most valuable feature to me; it helped us to start the performance testing activities much earlier in the testing life cycle. It also reduces overall cost, as we can replace the expensive third-party services with virtual services.

• Helped us start performance testing much earlier in the STLC
• Shortens the performance testing period, as there is minimal component-level dependency
• Reduced cost per transaction in case of third-party virtualization
• Reduced schedule dependency, resulting in more productivity
• Enabled parallel development for the development team

It is difficult and requires a lot of manual effort to debug/troubleshoot any virtual service. The overall UI must improve and must be user friendly.

I have been using this solution for around 2.5 years.

We did not face any major issues related to deployment, stability or scalability.

The technical support was excellent. Other than that, through CA communities you can get answers to most of your queries or challenges.

The setup was straightforward and there are many supporting documents available on the CA site that guide you through the process.

It is a very good solution as it covers most of the transport and data protocols. The process of creating a virtual service is pretty straightforward and easy. The only drawback for this tool is its license cost.