Broadcom Service Virtualization Reviews

The most valuable features are the web service harness and the service virtualization.

We have been able to enact a 'shift left' in our testing. In addition, I built an automated regression test pack for our teams. Thanks to CA Service Virtualization, we have new functionality (the ability to stub services we rely on) and we can test our own APIs in an efficient manner.

The GUI can be hard to navigate. The ITR keeps pulling the screen to the front. The speed can be improved as well.

We have been using this solution for close to two years.

We did not encounter any issues with stability.

We did not encounter any issues with scalability.

Technical support was provided from our work team. It was sufficient, but the response time was slow.

Initially, we used scripts, JCL, and CICS transactions. CA Service Virtualization has simplified and enhanced this process.

The setup was straightforward. The development of the harnesses was more challenging.

Before buying the license, make sure that you try the product and are happy that it can provide the service testing capabilities that you require. Have people in your team who can invest the time to gain skills in CA Service Virtualization.

Our team did evaluate other products, such as SoapUI and Topaz.

Make sure you understand the potential of the solution and that it will take time to make it work.

• MQ virtualisation
• Web service virtualisation

Time-to-market has been a lot easier and quicker. QA can start automating from the beginning and developers can do unit testing faster. Third-party integration is not required so you save money by virtualizing third-party services.

Integration with Selenium and GUI testing needs improvement. Supporting more new technologies and existing technologies such as Flex, Salesforce, etc. would also be nice.

I've used it for five years.

There were problems where it doesn't support the technology. We have to write custom functions ourselves. This means our licensing cost is more higher and makes it harder to sell our products to small vendors.

There were no issues with the stability.

There have been no issues scaling the solution.

Customer service is good.

Technical support is good.

I have worked with SoapUI Pro, HP ServiceTest (now UFT), IBM RIT, etc. This product was taken by a customer and it was one of the best products for virtualization.

It takes some time, but after one or two tries it becomes easy.

We used a vendor team who performed nice work. If they supported new technologies it would be better as it's expensive.

Over about a year, we have saved over half a million for our customer.

It's a good product for virtualization, but the price is high although we won’t regret it for this product.

Quick setup of virtual services

Creation of virtual services by simply providing XML request/response pairs

We have used service virtualization to quickly spin up training environments. Recently, our dev teams have started using service virtualization to improve the times of the Continuous Integration Testing (CI). Using service virtualization has reduced the time for each CI cycle from 4 - 5 hours to under 1 hour. This allows many more CI cycles to be executed per day and per sprint allowing quicker time to finding bugs, as well as faster times to promotion of builds to the test environments.

VSEasy is a nice feature but it needs to have some catches added to it to keep from inadvertently overwriting someone else's virtual service.

3+ years

No issues with deployment.

No issues with stability.

There was a memory leak in LISA 7.0.3 that would eventually require the rebooting of the memory to free back up the available memory. I have not yet been able to test DevTest 8.01 at that level to see if the memory leak still exists.

CA Support has been great for me!

Very good!

CA Service Virtualization (Virtualize) has been used to a greater degree, and Test has been used to a certain extent. The clients I have worked for purchased CA Service Virtualization with the primary goal of virtualizing web service calls to a specific back-end functions that faced constraints (performance, availability, and/or data constraints). By using CA Service Virtualization (Virtualize), they were able to overcome the stated limitations.

In the performance test of a middleware application, we were able to run a much higher load with CA Service Virtualization which some of the real endpoints cannot handle. 

• By achieving this, you can better measure and get the performance throughputs of the System under test (SUT), as well as, narrow down the bottlenecks with a greater precision of the SUT.
• In one of the environments, we needed a 24/7 application availability, which was not realized when the middleware application was pointed to the real backend. This was due to one or more of the real backend being down on a consistent basis. Pointing to CA Service Virtualization services, this goal was almost achieved.
• Eliminating the test data constraints: When testing against various backend, sometimes you need to test with a specific test data (such as a valid Credit Card number) or your test data needs to go through a series of validations in order to get a good response from a subsequent web service request. Using CA Service Virtualization validate, this constraint is no longer an issue.

I have used this solution for a little more than 2 years now.

We had issues in the support for virtualization of JDBC connectivity. Also, using Java Agent virtualization was a maintenance overhead in an earlier version of CA Service Virtualization. I am not certain if this has been improved in the current version.

It was very straightforward.

We initially used CA consultants to extend their product to handle some of the data protocols that could not be handled out of the box. The CA consultants did help in doing the initial setup. After they were done, we were able to spread the services and increase the scope of usage on our own.

Without trying other products in the service virtualize space, I feel very comfortable working with this product. It does its job well, is not difficult to use and grasp, and the support is very good. I would definitely recommend this product!

We were using this product for service virtualization, and SoapUI/REST API testing for the US’s largest retail customer in the home improvement space.

For service virtualization, we had a lot of subsystems in our application; and we had lot of dependencies on the downstream applications. We extensively used service virtualization to reduce these dependencies, which could be due to readiness, or downstream applications having downtime. We also had few third-party subsystems whose technical support was not available during US night hours. This resulted in a complete loss of the offshore effort in India. This was hampering our release dates, which are very critical in the retail business. Missing a release has a directly proportional impact on revenue.

For web services testing (SoapUI/Rest API), we initially had our web services test cases automated using a custom built Groovy/Selenium/SoapUI framework. Though the framework was robust, it was difficult for us to maintain the automation scripts. When our customer bought CA Service Virtualization, we were able to move our 3000+ automation scripts to Service Virtualization scripts within 6 months while supporting new features and producing regular releases. This was commendable.

Training our folks on Service Virtualization was also pretty easy. We hired three Service Virtualization experts from the market, and were able to build a team of 14 Service Virtualization experts within three to four months. CA Service Virtualization is a very robust product and is highly configurable. This helped us to take more user stories, compared to the days when we were using our home grown framework.

To the best of my knowledge (and memory), listed below are the few challenges we were facing at that time. It could be possible that CA would have already addressed a few of these limitations and short falls.

• Simultaneous execution of automation scripts was not possible; or, at least, not readily available. We had to write some wrappers to handle this situation.
• Integration with Jenkins was not available. We had to write small utilities to achieve this.
• Integration with JIRA was not available to automatically log a defect if there were script failures.
• It consumes more memory and makes the system very slow, which was stopping simultaneous execution.

I have used this solution for more than two years.

As mentioned in the room for improvement section.

I don’t remember any issues as such, apart from the ones mentioned in the room for improvement section.

As I mentioned, we were using a home grown Groovy/Selenium/SoapUI framework. Once a customer bought the CA Service Virtualization license, we were forced to use this. However, we found the product to be more robust than our framework.

Initial setup was pretty straightforward in terms of installation and configuration. Later, as we encountered a few challenges, we built a couple of utilities.

It is pretty expensive. On the licensing part, we don’t have much insight because the customer was handling this part.

We had evaluated multiple frameworks because we were working in a multi-vendor scenario. Every vendor brought their frameworks and solutions to the table. I don’t know if the customer evaluated any other commercial products. A few of our team members had already worked on iTKO/Service Virtualization on other projects; and they provided good feedback on the product.

I would recommend this product for clients who are heavy on web services. This product is very easy to learn, implement, and configure.

The ease with which we can create virtual services and view the flow in the model are valuable features of this product.

CA Service Virtualization has enabled us to give the client demos in a very effective manner even when the environment is not available. It has also enabled us to shift-left.

There should be an option to search a transaction that is VSI based through a transaction number. It would really be a great addition.

I have been using this tool for three and half years now.

Stability issues do arise if there is not enough system memory.

We have not encountered any scalability issues.

The technical support is good. I would give it a 8/10 rating.

The initial setup was quite straightforward because the steps to install and configure were clear.

Pricing is a bit on the higher side. Owing to its high cost, proper business justification needs to be provided to the client, if this tool is to be proposed.

We evaluated another solution, SoapUI NG Pro.

I would recommend others to use CA Service Virtualization for all their virtualization needs as it is easy to use and the support provided via the CA Communities is also great.

• Web service virtualization
• MQ virtualization

These two features helped us to build a virtual development environment for continuous testing and continuous development of our product.

The release cycle in our project was initially three weeks and we wasted time waiting for environment availability, because the dependent services were still under development. In Agile, we need to get the test environment up and ready before the code completion.

With the help of virtual services developed by CA DevTest/Service Virtualization, we were able to create the virtual environment.

Now, the release cycle is reduced to two weeks, with a fully developed virtual service environment.

We developed virtual services for the HTTP and MQ protocols.

Even though there is no comparison with its peers and all the necessary things are right in hand, there is room for improvement with the Virtualizing Mainframe components. The concept of agent installation is a tedious process and should have been much simpler.

I have used it for 5+ years and counting.

I have not yet encountered any stability issues. I mean, I don't think you will, if you have right configuration.

I have not encountered any scalability issues.

Technical support is 8 of 10. The community forum is the best of its kind and we can find solutions to all general issues.

We previously used SoapUI Mocking; it is simple static virtual services. For dynamic behavior that takes less time to create (hardly two minutes), DevTest is the ultimate.

Installation and setup was very easy. It is easiest tool in the market to learn and implement.

Compared to the initial license cost of the product, the cost has been reduced a lot and it is competitive. Of course, iPhone does not match Nokia on price, as the price reflects the quality. Nonetheless, the price for this product is very competitive.

Before choosing this product, we evaluated SoapUI Pro, IBM Rational Test Virtualization Server/GreenHat and Parasoft SOAtest.

Just go with it, if you don’t have constraints regarding price; it is the best. Otherwise, for simple virtual services, go for SoapUi Pro. Rule out the others.

The web services protocol for SOAP requests and responses is the most valuable feature. It is easy to set up the test for them without much coding, thus it is easy to train new testers to work on it in a graphical development environment where one can see the flow of information visually like the TIBCO product. It is easier than SoapUI to follow.

This product has introduced a simple-to-use middleware tool, which helped our team conduct web-services testing.

The most important thing is to have integration with Excel. It has a way to read the Excel sheet but not to write onto it. If I need to generate customized reports in LISA in Excel sheets for management purposes (which can show me number of test cases pass/fail/not started, etc.), I can't do this since you can't ask Service Virtualization to put specific values in particular cells with a particular color, etc. We were able to achieve it only using Java, which meant a lot of scripting, error debugging and rework, and this defeats the purpose of it offering a codeless environment to testers. A simple groovy scripting in SoapUI can achieve all of the above.

The Service Virtualization section needs to be more intelligent in offering a good solution to customers. For example, the virtual service created via the request response pairs needs spoon feeding of responses before the service starts performing. And I am not sure if it serves the purpose of virtual environment till the actual system becomes available. I have yet to see a proper implementation and successful usage of a virtual service through it.

CA should provide a free trial version of the tool (was not available till version 7.0.10) so that clients can try it out before making a purchase.

I've used it for two years.

There were no issues with the deployment.

The product is yet to become stable. I have seen instances when the tests have not been repeatable in LISA when scripting is done in Java. After removing and adding a character to the code, the test starts working again. Sometimes restarting it gets it working. Also, there are caching issues in LISA and it picks up the old values of the variables/parameters/properties in the tests even though the values have been updated and saved in the test before commencing the run. Thus, people prefer using more stable solutions like JMeter and SoapUI.

There were no issues with the scalability.

The customer support for LISA is average with the associates not being reachable all the time. Also, there were no specific communication protocols set up to interact with LISA support team and we were mostly relying on the Confluence forum to have our queries resolved. Also, the search results on Google do not provide answers or hints to the questions LISA testers face, unlike SoapUI and Ready API which have a lot of online support available. One can only rely on the user guide of LISA which does not cover all the scenarios.

The initial setup was straightforward, but the settings procedure, like providing the correct domain name, server name, username and password, did not work the first time they were provided from LISA.

I got it implemented through a vendor team. There are a couple of settings you have to play around with before it starts working.

Choosing this product was a management decision and we would prefer other solutions because in order to achieve a customized solution one would have to do scripting. This tool needs to offer a lot more options before it can be marketed as a codeless environment that can really help an organization chive results with such a testing team that is not good at coding.

QTP can achieve almost everything to do with scripting testing. Other tools like SoapUI/Jmeter being more popular become the obvious choice.