Broadcom Service Virtualization Reviews

Using the agile method, each iteration is scheduled to last 1-2 weeks and development would take up most of that. The testers didn't have enough time to properly test the application. With Service Virtualization, we virtualized the services and we brought quality into the agile iteration “extreme left”, and we started early testing.

That's how we got enough time to test the product. Service Virtualization helped us by using marks and stops. We are easily integrating with other applications and we are performing integration testing seamlessly. Before implementing Service Virtualization, services were not always available because of infrastructure issues or iteration time schedules. Service Virtualization removed all these problems that we were facing in the industry and allowed us to quickly do our job.

We not only use Service Virtualization for web services, we also use it to virtualize services that we used to pay for. It can be used in the enterprise bus to integrate external applications. In the QA domain, you need to pay a lot to use their services. Service Virtualization can save you this money.

For example, I have previous experience with the financial and insurance industries. In the financial sector, we integrate with third-party credit agencies to check credit scores. You still need to pay for these services even when you are only testing your application. If you virtualize that functionality, you don't need to talk to them, and you don't need to be dependent on their system availability.

We still have mainframe systems and while CA provides instructions on how to integrate Linux-based applications, they don’t provide integration steps for mainframe.

We would like to see improvements in mobile integration. Integration is partial, but we would like to have full-fledged functionality. Android mobile solutions are out there, so you can download an app and have full-fledged mobile coverage. You can set the model and Android operating system. But we can’t integrate these with development testing. If we get that feature, we don't need to look at other products like HPE Mobile Center, or something else. Having mobile would be good for us.

We are also looking for more knowledge to share with our developers, and others, on how to use Selenium scripts within CA Service Virtualization. The process is pretty simple. You build everything in Selenium, convert the script to JSON, and dump it to Service Virtualization; alternatively, copy the script and put it in CA Service Virtualization steps. But these guidelines are not clear enough. Clarifying this process would definitely be a benefit for CA.

Service Virtualization is pretty stable.

We are in the financial industry and process a large number of transactions. Every second, we process around 78,000 transactions; every second counts. Sometimes, especially when the servers are down, our message queues will accumulate hundreds of thousands of messages. When the issue is resolved and the gates are open, all those messages will flood the system. We tested all these things with CA Performance Testing. In real time, we cannot test using Service Virtualization, but we built all the messages, and the message queues, and we tested by opening the gates. We found that scalability is also pretty good.

Calling technical support works out good. But the website search function could be improved a lot. Their knowledge management system is not up to the mark. The search functionality really needs to be improved. CA has to really spend some time looking into that. Depending on the search, the content can be very limited. I would like to see examples and links to the documentation. The current search function is not detailed enough.

I'm a systems architect. I had a lot of experience with the development infrastructure architecture. I see that there are problems in all production areas. I always do research on the latest and greatest products. CA is one of the companies I've been following for almost 15 years. I discovered Service Virtualization when I learned that CA acquired ITKO.

I was involved in the initial setup at a different client. It was pretty straightforward, I never faced any issues, and in fact I purchased myself a license from CA, spending a lot of money on this. I was a good investment for me. I'm very passionate about technology. I have bought a lot of software, but it is worthwhile spending money on this.

We also use HPE for testing right now, particularly Mobile Center. Not every product will address 100% of the issues.

However, CA Service Virtualization is totally number one in the market, no one comes close to this. HPE introduced a product recently, but they're nowhere close. They need to spend a lot of time on that product.

Definitely look at the capabilities of your application initially. Everything depends on that. I can say that Service Virtualization really helps you to reduce time spent testing, and improves the quality in the agile process. If you are not using this, I advise you just have a look. Definitely you'll like it, and you'll implement it.

It helps us to create service scripts for performing shift-left in our testing.

In the development environment during the early lifecycle, whenever there is a third-party interface that is not available, we use this product to create those steps that help us in getting responses, as and when needed.

It has helped us to shift-left in our testing life cycle.

We do start testing at an early stage in the development environment, even before it gets into QA.

Generally, for our agile development projects, most of the third-party interfaces or interfacing applications which are shared services such as middleware services are not available. This is when we use CA Service Virtualization for early testing. Once we get into the QA environment, we test with real interfaces.

This product provides an interface with current test management tools and that’s what we like about it.

However, we would like to see more interfaces with all open source platforms, especially in the CI/CD framework such as Jenkins, TFS tools.

It is very stable. We've been using it for quite some time.

This product is very scalable.

I have not interacted directly with technical support. We have our own team that is in charge of this aspect.

Previously, we did use HP Service Test and SoapUI software, but we find this solution to be more scalable and stable.

I was not involved in the initial setup process, since we have a setup team that takes care of this.

Not many people know that even in the development space, developers can utilize this solution to perform some of their unit testing, before they get into real interfaces. This awareness is not there in the market. We use this product in performance and functional testing, so probably even in unit testing the developers could use it. Thus, we would strongly recommend to explore this tool and its functionalities.

The most important criteria while choosing a vendor is primarily its performance track record in the market and the range of solutions they provide across lifecycles. Another important aspect is the ability to interface with multiple solutions that are available in the market.

The ability for us to be able to record the live transactions, and create a virtual services out of that, as well as be able to actually use the request and response to create new virtual services for the back-ends that we don't have access to. We have a bunch of use cases there from that perspective.

With the radio's hardware costs allow the development team to what they call shift left, and start to develop their code against the other back-end services that we may not have available in time. These are pretty much the benefits there that we see.

I would like to see more integration support with Docker for release automation, a few things on service monitoring that's pretty much provided in the prior version of the product. I would like to see that a little bit more usable in the future.

It's really good. We use it to support performance testing as well. That can scale up to tens of thousands of users concurrently.

It's great. Basically, it's designed so that we have a load balancing virtual service environment, so we can scale it up as much as we need to.

That's pretty much the key thing I like about CA compared to other vendors I work with. Their support is great. We have an account manager that's really helpful. He helps us kind of navigate through a different escalation path as well after we open the ticket. We never felt like we were left hanging when we run into any issues with the product.

I think it's a little more technically involved to get it started, but once you get the hang of it you can pretty much do even internal knowledge transfer to any other team. We rely on CA-provided training to a really small group of our team, and then after that we first do our internal training and knowledge transfer to other teams to scale it up.

We quantify what kind of benefits that we have against the cost that we have to pay on the solution. It turns out to be a really good exercise there. It forces us to do the ROI on the technology product that we buy against the benefits that we reap from the product there.

I would rate it an 8/10. From a capabilities perspective it's there. Like I mentioned, the learning curve can be a little high. Usability in the prior version that we used, version 8.2, usability is not great. That's why I give it an eight. It requires a Thick Client, so we need to install that on the machine. It's not the most intuitive UI ever, so we'd like to see it improved a little bit from that perspective. I know that they're coming out with version 9, which is supposed to improve that quite a lot, so I'll be looking forward to that.

What we start off with is that we had what we called a enter of excellence that comprised of a few very technical individual to try out the platform, to install the software, and be the product expert within the organization. Then what we did is that we pretty much scaled the development of the virtual services to other teams and that have more of a knowledge of how the service is supposed to behave, so that the COE doesn't need to be in all of these type of sessions and what not. We actually used the start out small and then use that team to empower other teams to actually be able to develop their virtual services that way.

Pathfinder, VS Easy

Ability to implement WSDL validations in the model using the built-in assertions.

The virtual services for Stress, Volume and Performance testing for applications which reduced the cost and also eliminated the dependencies of other applications and third party software.

Not all protocols are supported, but I've heard there will be a feature in version 8 with the help of which we can create models for those scenarios as well (surely a few limitations will exist).

3+

Sometimes, different versions of tools differ in behavior and new versions don't completely provide backwards compatibility.

Example: lack of ability to deploy a service built in LISA 7 to LISA 6 VSE using LISA 7 workstation.

When creating services with database steps, if the DB goes down the services will end and once the DB is up the services will be still be in an ended state, and a manual start will be required.

Triggering multiple requests sometimes increases the response time even though loadbalancers are used.

Customer service is very good.

Good.

We were using LISA 6, THEN upgraded to LISA 7.5 as the new features were beneficial for meeting business requirements.

Initial setup for Workstation is straightforward although setting an agent and broker for pathfinder to use recording is bit complex.

We implemented through a vendor team.

We have eliminated the complete dependency for all the backend systems for SVP (Stress,Volume and Performance). Testing using Service Virtualization saved huge amounts. Using service virtualization we have implemented a shift left concept to enable different teams to work in parallel instead of waiting for other teams to start the work, this also reduced the overall budget for the program.

A few other options were considered from Parasoft, HP and IBM.

For CA Service Virtualization, it’s the ability to quickly prototype something. My guys really like the ability where they can do the recording session. It's a way for them to initialize for existing services where they need to get it up and running. The ability to have a listener and capture sample packets was a key thing that they really liked. It really helped us jump-start something. We could do something within an hour to have it up and initially runnin

Also, the ability to have the different back end connectivities, whether it's an Excel spreadsheet, or more complex things where we’re now linking the data sets and responses back together. Those have been a couple of the key areas for us and have been very beneficial. It’s also certainly a lot better than just doing stub code, because now I have templates that I can more readily reuse. That's better than just somebody who’s kind of building up the Java code to build stubs.

We keep running into situations where people will start building stubs and things like that, and we'll come back and show them the benefits of this solution. Once they start to see it in use, then they start to say “oh, okay, this is a lot better on that side.”

From a CA Release Automation perspective, it’s certainly the idea of being able to do the automated deployment. The challenges are that we started off down this path a few years ago, we purchases some licenses and have not taken full advantage of them to this point. We've had an ongoing challenge within our environment that stood up quickly.

Now we're getting a little bit more focused about this. I'm already doing some work on it and we’ll be doing more over the next few weeks. We’ll be looking at the Release Automation tools for coming up with the best processes for us to be able to have a repeatable process that quickly can deploy code without having to do a lot of manual steps. We want a good and clean workflow.

I think one of the things that we did appreciate was that changes were coming in to the product at a good cadence. We needed to support WebLogic, which was a big one for us at the time. Those things did come in, and we didn't have to wait a huge amount of time. I always felt like the product has been getting good updates to support us as we were doing some of those activities.

One of the things that we know and we're trying to work through is looking at when I go to set an environment up, maybe I don't quite have that new service yet, but I have other applications, my UI application, or whateveris ready to go, but I have this other middleware call that needs to be available. The idea that I can spin up or point myself to a virtualized service without one piece of it, but still use the rest of the end to end system, that's kind of one of those things that we would envision.

I'm doing a deployment of an environment / application, it's being configured, and then if I need to I'm going to use virtualized services for some or all. What we've been working on is how we can do a lot of that shift to the left by using service virtualization, so when we deploy we can at least get the development teams testers up and running on the application. Then we just have them virtualized on the back end. That might be how we would be setting up and configuring ourselves. It's definitely in that situation where you don't have a true end to end environment, but still need to be up and testing. That's where Service Virtualization would couple with the deployment in my book.

CA Test Data Manager is where there's one kind of end to end system and there has to be a system of data. A lot of times these call systems are hitting, and so having that model so that we can get it into the data, so when we deploy the software, you're going to bring this up. It may be our building data, which is an interesting challenge for us because it's a third party product, though they work very closely and have people on site with us.

Other systems that might need to have a configuration- there's all the types of users that are allowed into the system, or other types of price catalog information, whatever we want to model, that's where TDM is in your standing open environment, that's where you need to have something in place.

For some of our systems where we have to have data in it and available to product catalogs or something like that, then the TDM data can be very beneficial, and we can swap the data around, so we might be trying a new product catalog that's coming out, or new few features that are going to be offered, whereas we can also then go back to a production-like configuration as well.

When I originally bought Service Virtualization there were several things that we leveraged it for. One of them was that we had a merger acquisition, and we needed to interface the two systems together. So we needed to be able to share all the account information and those types of activities that were happening. There was going to be a set of middleware services that were going to be built on our end to allow the other system to communicate. Before that was ready, they wanted to start testing. We could use Service Virtualization in that case to let them start working with it before those middleware services were built. That was really beneficial. That's actually how I brought the product in in the first place.

The next thing that we really used it for was training. In the past, we tried to have training environments that were done different ways. They actually tried to have one application where they had two people dedicated to just trying to keep a full end to end environment up and functioning. They were never really able to do that. It kind of went by the wayside. It was expensive, and they just couldn't keep it in sync, and it was breaking. You've got enough different systems going that they couldn't fix all the issues. You had to go to either the development teams or somebody to help you out with how to figure out where the problem was. That wasn't real tenable.

Another way people have been trying to do it was they would build a simulator. One of them was a Flash-type simulator that tried to do it, but of course that would get out of date, and then they'd have to go back and spend a lot of money that way to try to update it. What we did is we built the services we needed to allow the applications to work in a training scenario so people could run those training scenarios when new sales people were coming in. They could run them with our call center folks and they have an opportunity to start working with the app ahead of time and not have to work with real data, per se.

We've had a situation where folks forgot to scrub the phone numbers. They took them straight from production, so a little bit to the TDM type of situation, and what happened was I got an e-mail shortly after we released one of our products, it had come all the way down from the CEO, people were trying to figure out what happened - a lady had called and said her daughter had had her phone number changed three times in the course of one week.

It turned out that people were actually using the real live system instead of the training environment, but they were reading the training documentation, ‘do this, put this phone number in.’ Those phone numbers were real phone numbers in this case, and we would treat it as one, or we'll call it real account numbers. People were changing those account numbers on the customer, and that wasn't so good. We quickly went to scrubbed account numbers, but that was one of those things, just one of those side effects where a TDM type of solution can become very important and helpful to make sure that you're running it through that kind of scrubbing process.

Those are a couple things, the training, the helping to get things going, and then the shift to the left. That's been a real benefit for us. We can get more testing in sooner and we actually had one project where I pushed us to use it because we couldn't get a stable environment, so I said let's virtualize those services. In doing that, we actually were able to allow the teams to do all their testing. By the time it got to our QA testing, I believe they found only one defect in the application on that.

The later in the cycle that you find defects, the more expensive it is, so finding those things upfront and doing it even without an environment that had full end to end capability, which was kind of always my point, was you may find some integration issues, but you can get so much of the functionality tested of an application with Service Virtualization if you've done a decent job versus just having to wait to get a full environment, especially when you don't have stable environments.

For Release Automation, obviously what we wanted is things that are repeatable so that we can do things faster, and it doesn't require the manual configuration because when you start to drive for faster cadence, you just have to rely on more and more automation. It has to be a known process, you implement it, and if you fix it once, it fixes it for all of your deployments. The other important thing here is that you must make sure that whatever processes you settle on, you're going to do it. Whether you're doing it in your Dev environment, your QA, your staging or your production, you want the same process everywhere so that if you find the problem, you can fix it once, and that fixes it for everybody. You’re always running a prod type environment as well, which just ensures much more when you get there.

A lot of companies including ourselves have been in a situation where what we do in production is distinctly different from any of the other environments, and that leads to a lot of extra resources being dedicated just to managing the production environments. Release Automation allows us to implement processes that can be reused through all the environments.

Even if you're not getting a full DevOps model in place yet, it’s critical that your production support people are working with you up front beginning with the development teams, which is how I actually started promoting the concepts. The concept of DevOps was because we needed to be able to really get in sync with what we wanted to do in production, as that was the end game. We needed people to be much closer together in alignment. That development was building things that were appropriate for production, provided the insights into the stability of the application, things like that. At the same time, we work on getting closer to a production-like environment all the way through.

There were things early where we couldn’t do a few things in Service Virtualization that have since been updated. The concern I have right now with Release Automation is the concept of what we call immutable objects, so I build it once in development, and then I move to all of my other environments.

My challenge is looking at this from a perspective of “I'm kind of looking at … do I look at containers, do I look at what the RA type of product provides?” I also have to consider being cloud ready. As I deploy into an environment, I want to make sure I have enough of a stable and a repeatable environment type of model where once it comes out of development, I know that everything is staying the same.

There’s different ways we can slice it, and depending on whether you do containers or a combination. That's some of the things I'm trying to understand more with Release Automation right now. I've been pretty happy with the product, I'm sure there are always things that we could ask for, but it's really done a nice job for us.

It's interesting that it still takes teams a little bit to wrap their heads around. They're so used to kind of stubbing. I feel like the product has been able to scale with us, I think more of my challenges are how best to handle the processes. For example, who should be responsible for virtualizing services? I think that generally, the best thing would be for the middleware folks who are building a new service. They should also be responsible for creating the template virtualized service, and you build it right up front so people can start using it. I think there's a part of it where people could then also modify and update data, so there could be other types of responses for their scenarios, and it doesn't have to be done by the middleware folks. We went with a model of a center of excellence where they know the key, and then we start trying to educate and train other teams to build out more. Initially, we were not to that spot with the middleware teams for a variety of reasons, so we had to rely on the individual teams to build the services.

There is what to be careful about - let's just say I have a customer lookup call, I really don't want to have to end up in a situation where two different UI teams who need to use the same customer lookup call did their own virtualizations of the same call. Those are some challenges for us that we've got to figure out. That's more procedural than it is limitations of the tool.

We have had no issues with the deployment.

The performance one - we want to start hitting it a lot more frequently. We've used it for some of the performance testing, and even the development worked pretty well for us. Even with all the training environment things we've done, I don't think we've really bumped up against a usage issue yet and had to put another instance out or whatever else we had to do to opt the number of times a hit could come in. There have been some issues with Service Virtualization, but I would have to can ask the team for more specific details

We just haven't ramped the Release Automation up high enough to know. Certainly the concepts of how it's designed seem good. With Service Virtualization the performance has definitely been handy in some spots for the kind of capability. Both of them have been scalable, I'm not aware of any real concerns regarding the scalability.

We built a solution in-house that coupled with data, the configuration of endpoints and how the app needed to be configured in a given environment. We had it coupled with ServiceNow to do workflow, and then trigger the builds, but then we could also deploy, trigger the deployment of the Release Automation product and kind of integrate into that whole flow.

There hasn't been an issue per se from Service Virtualization. There are areas that I want to dig into more – containers. The idea about how can I quickly decide what to do with Service Virtualization in the way of quickly getting it into an environment so that if I need to hook up to certain ones or certain applications because maybe that it's a third party that doesn't have a test environment, or whatever it need to be, maybe it's a credit check call or something like that to hook up, how can I quickly in spinning up an environment leverage the systems. That's something that we'll need to look at from a deployment to get that more integrated into our whole environment process.

Regarding Release Automation, I know that there were definitely some good strides and improvements that happened to that product from the first days of Nolio to where it is now. I know that especially before the WebLogic support, some of those capabilities.

For Service Virtualization, we went ahead and went with the CA recommendation of having some initial consulting to help us get started and help get the guys kind of going up to speed. What was interesting for us when we did this was the consulting guys come, and we were able partly because we had this merger and acquisition going on. There were some funds to be available where we might not have always been able, because CA rates are kind of up there.

It was kind of funny, we had already been working with the product for three or four weeks on our own, and we had already done some basic virtualization. I remember when the consulting team comes in, and the project manager says, "Well, normally, we'd be working with you just to start with how to get your environment set up and all that, but since you guys have already done all that, we can skip past all of that." We dove right into best practices and techniques, which I think the key is, and it's something that I have in my talks that I've done at CA World, I leverage developers to do my virtualization to start with, because what I talk about with developers are that they have a broader set of skills and bag of tricks that they can do.

If they look at a problem, and they have this, maybe they have some data set sitting there in some file, and they can't get it over into the other tool the way it is. They might run a little simple application to translate that data into a way the other tool recognizes. It’s the way that you think about design patterns and implementations. I had a team that started off with four people, and then we expanded to six at one point. We were leveraging some offshore resources as well. We were able to make headway quickly by just actually bringing in some fairly technically sound developer types in to do it, rather than people from a QA organization, where it can be a little bit more of a challenge because you don't necessarily have all those development skills and capabilities that a developer has.

I was able to get a good set of resources in and developers to work on it. One of the guys switched what his role was and became the lead of this effort. Because of all his skills and background, he got there quickly. We didn't have a lot of problem ramping up. I think CA did help us in the sense of jump starting it, but like I said, I firmly believe in the use of developers if you're building a center of excellence because they're just going to give you some of that robustness and richness in what you develop that is probably going to be more tailored for reusability. They're going to be keeping an eye on it, or at least that's what they should be doing from that perspective.

I think there's some things about the licensing and for putting it onto every sever, that's one of the things that will definitely be an evaluation point as well as we look at solutions and how we might do things. If we have to have one of those, if we end up putting it onto every server, pricing will be a bit of a question.

Where I see Service Virtualization and Release Automation coupled is at the point of some type of environment testing. I'm going to be doing testing, so I'm either doing my testing in development, or I'm doing my testing in QA. Those are the most obviously common places we're going to be doing that. There's integration testing, but I don't necessarily have all the systems readily available to me for function up and running. Or sometime we literally might have an outage while they're updating the middleware call, so maybe it's important for offshore to be getting their testing done.

The ability to swap in or hook in virtualized services so that you can keep your testing going is important, so when you're doing the deployment, you could have your setups configured such that you say whether or not you need to select a given endpoint to be pointing to a virtualized service or the live service. I think it's important that you know how that data plays together, and which systems or maybe there's a group of them. You can't simply change one service. Maybe I have to have all three of them virtualized, running all the virtualized ones in one situation because of the way the data works.

This whole idea that you don't necessarily have to have a full end to end environment to get people functioning is important. 

What’s been the key is to get people embracing it are seeing what Service Virtualization can do for them.

Regarding the virtualization capabilities, I remember when I saw them and I thought I knew exactly how I was going to use it. Training was one of those things that I took to the business that actually triggered a statement by one of our sales operations people who said this changes how we hire people. That caught me off guard, I didn't expect it, but it made sense once we talked about it a little bit more.

I'm a software developer, I just want to get in and start working with it. I just wanted to get my hands on it. I want to work with it, touch it, and feel it. 

When discussing changes in how we hire, we always had to think about all the type of training that they had to do. There's weeks of training as you're hiring people, but you don't always have the time to spend four or five weeks training people. The ability to have those training disciplines right there, and not having to maintain a whole end to end, which is almost impossible, and you're never up to date with that type of scenario, that's a huge win. Of all the products, I think Service Virtualization is the easiest one to sell and the easiest one to get your arms around as to why there's a benefit, and I was able to sell it quite readily into the company for those very reasons.

One of the things that we just have come across here is that you setup a training environment, but depending on how a company does their projects, if you're going to start something and put training in it, then you need to make sure you also stay committed to it so that it get enhanced. You need to work closer to the business, so that they have a sense of the value, and you need to make sure that it's factored into the project work. A lot of the time development teams just think about how to code new features but we also need to worry about operational, meaning whether I'm relying on my release automation tools. Questions like, how do I integrate, how's my apps being monitored, how easy is it to test them, if an issue comes up, how quickly can they detect it?

There's so many good things that Service Virtualization can do, and almost everybody suffers from the same thing with middleware teams who aren't responding quickly to them. You’re on different cadences of when things can be released, and just trying to have stable environments. Service Virtualization can help work around some of those challenges.

It's the ability to capture and replay traffic that's most valuable to us. We can monitor network traffic while getting data, and a virtual endpoint lets us see what the platform would respond with. We use it to isolate our app from unreliable pieces of the ecosystem, protecting our code from uncertainty.

Isolating the application that we’re testing is the top benefit. Also, it adds the ability to generate predefined responses to data so that we have a complete understanding as to how we’d perform.

So far nothing, it has everything I've needed. Totally everything.

However, it's a pretty heavyweight tool for my personal needs. I like to do stuff on my own and would normally prefer to use my own solution for service virtualization, rather than such a heavyweight, extensive product. In half an afternoon, I could make a few scripts to meet all the needs myself.

I haven't used it for long. We went through training, and have been planning how to use it as our app is still being developed. Training usage was two months.

It seems to be scalable, but we don’t know to what extent. For my testing needs, the solution has been fine, but I’m not sure from an enterprise perspective. I’ve been at Nike for two years and used to be just a Java developer.

I've not had to use them.

We used Charles Proxy and it required much more manual work. This product is much more automatic.

This is what I'm primarily doing. It's been straightforward with no issues.

It was a corporate decision to change. They looked for a solution that most accurately solved the need at hand efficiently. The most direct applicable solution to my needs is the important thing.

Jump in and play around with it, and you'll get it easily.

The ability to virtualize test environments. This allows us to test on an environment without having to wait for time windows, thus eliminating a bottleneck in a traditional shared test system.

It eliminates bottlenecks in the test system sharing.

It is complex technology which takes a high end skillset to set up. You are very dependent on consultants from CA or a small subset of CA partners to implement CA Service Virtualization in the beginning. The training offered by CA is also not up to par. It is not realistic to expect that you can purchase CA Service Virtualization, send a couple of folks to CA training, and get yourself up an running with CA Service Virtualization environments without outside help.

I've been using CA Service Virtualization for more than three years

Yes, we had to thoroughly train and vet the engineers that we hired as CA Service Virtualization consultants. We were partners with ITKO, the original developer of CA Service Virtualization, prior to CA acquiring them. The ITKO development team worked closely with us for more than 6 months to get the team ready. Finding the right skillset was key, plus investing the time to train them properly. In the end we built a lab and the successful engineers gained experience through practice.

No, the product is very reliable.

No, the product allows for great scalability.

I would rate customer service from CA to be very low AFTER you have purchased the software. They are great before you buy, and disappear afterwards. However, I will caveat this by saying the CA Tech support for Service Virtualization is very good. Just don't expect any warm and fuzzy customer service.

We had a fairly mature testing consulting practice for more than ten years based on Mercury's platform (Quality Center, now owned by HP). We still use Quality Center extensively and have added CA Service Virtualization to our 'tool bag'. We did not purchase CA Service Virtualization as a replacement for any product we were using. We viewed CA Service Virtualization as a game changer, giving us the ability to add test environment virtualization in order to reduce test cycles for our clients.

It was very complex initially, once you have one or two people on board that gain experience setting up environments, you will eventually be self sufficient. Expect to use outside professional services for the initial implementation though. Don't let the vendor's sales people convince you to buy just the software, some training and "starter pack" services. The services and training will not be enough. A more realistic approach is to rent qualified engineers for the first project or two where you are going to leverage CA Service Virtualization. Send your own staff to training and have them work alongside the consultants for these initial projects. after a project or two, your team will become self sufficient.

Initially we learned through the vendor, which took 6 months and was expensive. The ITKO team was very good. However, many of them left CA after the acquisition. There are a couple of partners in the CA channel who are also very good.

It is hard to put a dollar figure on it. If you have multiple large development efforts and are sharing common test environments, CA Service Virtualization is a huge benefit because you don't have to schedule your testing based on when the test system is available. However, if you are a smaller shop with one development team who works serially, CA Service Virtualization is overkill.

This varies greatly across several clients. CA will offer a fairly good discount because they are motivated to sell right now. The consulting rates can be high, because the population of skilled, available CA Service Virtualization engineers is very small.

HP has it's own approach to test virtualization which we evaluated at the same time we evaluated CA Service Virtualization, but we found it to be behind in it's capability compared to CA Service Virtualization.

Work closely with a CA partner in their Service Virtualization channel to get your in house resources ramped up on CA Service Virtualization. Expect this to take time and require investment in professional services, training and mentoring.

Virtualizing mainframe systems. Reducing dependency on mainframe capacity and availability, which is a major roadblock for us right now.

The virtualization is primarily performed by eliminating the back end mainframe system. So we can read out our API traffic to the virtualized system. Basically, we don't have to use the mainframe while our service is virtualized.

It helps us with work/life balance. If we didn't have that CA Service Virtualization, we would have to run our performance tests during the night, because that's when the mainframe is least used. For our team members it helps them to maintain a better work/life balance, as they can execute their tests in the daytime so they don't have to spend the weekends and nights doing so.

To us, right now, it is going to be reducing the capacity limitation on mainframe, because we are highly dependent on mainframe capacity which is not easily available to us.

Regarding additional features for future releases, that is something we'll come to know when we start using it at full scale. But so far, we're pretty content with what has been offered.

In terms of improvements, I think a little bit more of use cases from existing clients. That could show that it's a workable solution used with other clients as well.

Stability is pretty good so far. We have used it across our organization in different teams. So the next step for us is to bring it to performance testing.

Scalability is the next factor we have to judge, because this system has to handle a large amount of load, so we'll have to see what the scalability is going to be. But based on the reviews and research, we think it's going to be as per of our expectations.

We have had business partners, account managers, come and do the PoC for us, proof of concept, so we have used their support. 

Support has been great so far. They have come to the table whenever we needed them. They have provided all the support in a timely manner so, so far so good.

We made the move because of a constraint that we have today. It's a constraint with mainframe capacity and availability. This is basically an opportunity for us to remove that constraint and that's why we need Service Virtualization.

We have completed the PoC, so I would say since we haven't completely set it up. So far, whatever the setup has done, we're satisfied with it.

We evaluated other options, I'd say lightly, very lightly, because we have used CA within our company from a while. We know that they are the market leader, we know there are probably not many competitors for the tool that we want to use.

When selecting a vendor what's important to us is 

• a relationship with the vendor
• their response to us in a timely manner and
• especially, their being established, having proven their solution with other clients.

I give it an eight out of 10, based on our experience so far. Once we use it at full scale, we'll come to know if there are any more improvements or challenges that need to be addressed.

I would tell colleagues to keep CA as the number one priority to research because they are the market leader and have the most experience, based on our research. I think they should be one of the top contenders for service virtualization.