OpenText LoadRunner Enterprise Reviews

The primary use case for clients is that they often have large application development teams and application development projects that they needed to scale. So, for instance, if they were developing a new banking website and they needed to check that the application that sits on that website was scalable from a few hundred concurrent users to many thousands of users, they could test the load response using LoadRunner.  

That is what LoadRunner does, it does the performance testing and measures load-bearing response.  

I think, for me, the most valuable part of the product is the way you can do the basic testing. You can create the test script and then simulate thousands of users very, very easily. Instead of having to have lots-and-lots of systems that would emulate users, you just needed a couple to emulate tens of thousands of users. So the scalability of LoadRunner while it was testing scalability is really valuable.  

To improve the product, I think the integrations could be a little bit more slick. It does handle a lot of great integrations, but then some of them can be a little bit clunky to implement. The integration with third-party tools needs to be stepped up a little bit.  

As far as other things that need to be added, it has changed quite a lot recently, and I have not had vast amounts of experience with the latest version. So I am afraid it would not be fair for me to go further in expounding on that question. Things that I talk about may already have been included.  

I was a presales consultant and so I was kind of a technical consultant as well. I was working with the solution end-to-end for about seven years. My main focus was not LoadRunner for the entire time, but I gained knowledge of LoadRunner and I gave presentations about it. It has been about eight to twelve months since I last did anything serious with it. However, I am still familiar with the product.  

The scalability is really very good. In some ways, it is the purpose of the product: testing by use of scaling loads.  

The initial setup is pretty straightforward. But I have got to say, having worked on-and-off with LoadRunner over a period of time, I knew kind of instinctively how to set it up after a while. In other words, in my case, I would say it was simple. On the other hand, I think it the first time I tried to set it up it was a nightmare. After that, it was easy because I learned a lot about it. If I had to score it out of ten for initial setup with ten being the best, it would probably be seven-out-of-ten. It is not really going to be super easy for first-timers to deploy.  

The deployment could take quite a while, even when I was used to doing it. Getting the software installed and running is pretty quick and that is not a problem. But creating the projects and creating the test scripts can take a little while. To get up and running and doing stuff within it, it is probably just around a week. Doing it professionally with the integrations and with all the correct testing scripts, it can take a month and more. It really all depends on the purpose and how you want to use it.  

My advice to people considering LoadRunner is that if you are going to use the product, use it as part of your everyday application development lifecycle. Do not just use it right at the end, because it gives you some great insights during the development phase as well as at the end. You will end up writing cleaner application code with it. So bake the use of LoadRunner into your full application life cycle.  

On a scale from one to ten where one is the worst and ten is the best, I would rate this solution as between and eight and nine-out-of-ten. I could be slightly biased, having worked for the company that sells it. But it is a very good, professional solution. With the latest updates, it is very comprehensive and one of the best products of the sort. Let's say nine-out-of-ten because there is always room for improvement.  

For me, it's the fact that I can train my team on a single tool. It really is kind of our Swiss Army Knife in that it can handle so many different types of protocols and technologies, and I don't have to train my team on multiple tools to handle each of those.

It's really given us a lot of insight into, especially as we've moved into DevOps now, incorporating that into our CI lifecycle. We can start the performance testing earlier on, carry the same results through, and have it as part of our release cycle.

Mobility, I think, is the biggest for us right now. We're really getting into mobile app testing, native mobile app testing, so native Android and IOS devices.

As far as scalability goes, we're able to spin up load generators to handle an incredible size of load, so really we're just limited by the amount of hardware that we can spin up for it.

We use a third-party partner to do tech support and it's been fantastic with them.

We have, ourselves, gotten to Mircro Focus on a couple of issues that have been actually escalated all the way up to the team back in Israel, I think that's where they're located.

That was fantastic. We actually had them, they came out on site with us, some of the developers for Performance Center, and talked to us about some of the ways that we were using it. Some of the features that they thought customers no longer needed that we were actually telling them were still valuable to us, and they wound up bringing back a few features that were on the chopping block.

I wasn't directly involved in the decision to invest in Performance Center because we'd been using it since it was LoadRunner, and I've been using it for pretty close to 20 years now. So that was before my time.

I was involved in it. I'd say it was fairly straightforward for us because we used our support partner to help us out, guide us through some of the pitfalls of the initial setup. But he gave us a two-week timeframe that he said we'd be able to get it set up in, and we easily had it set up within a week.

When looking at vendors, support is the big one. Also, ease of training people on the tool, and just the variety of technology supported.

In terms of advice, I would say the biggest choice is that Performance Center really is for a very large enterprise. So I'd say evaluate if it's really what you need for the size of your organization. But if it is, it really can support pretty much any kind of technology you throw at it.

Performance Center is really our go-to tool for anything that we have to test. It's just our default tool for whatever technology we have.

Initially, I've been using it for small use cases, just to test scenarios of less than 1,000 users. I think generally it's been very good. My team has even deployed it for clients within banking. It's still a go-to tool; although, as far as SaaS goes, recently we have had more suggestions to go with Neosyde. 

I think the number one feature everybody likes is the capability to easily generate virtual users as well as the reporting. Recently, we are starting to look at things more from the diagnostic perspective as well as from the troubleshooting perspective. It gives us many more options for troubleshooting and presenting reports. The other reason why LoadRunner is quite popular for us is that it has a long track record. We know if we need to look for a solution we can still search and find a use case or a solution quite easily.

I like the new pricing model. It helps us to ramp up much better, especially when we were trying to use this for SaaS applications. They have a much more practical pricing model now. It allows us to break it down smaller and also build-up towards a price model that works for the client. I think that was a big bottleneck in the past — now it looks much better.

From a technical perspective, LoadRunner has always been good. You can trust that it can deliver. The big bottleneck in the past has always been the pricing model. Now, with the new approach, with the use of SaaS, we are currently in proposals to recommend LoadRunner as a solution for one of our government clients. We are doing an implementation there. 

I think LoadRunner is still getting into grips with me — maybe, I've not used it that much. It's not that popular on the cloud. Also, we have not tried this on mobile platforms with mobile virtual users.

I have used LoadRunner for quite some time — roughly 10 years.

Both the scalability and stability are strong points for LoadRunner. We have no complaints so far. Of course, there's always this concern around if we have sufficient use on the hardware to create the required scale for the number of users, but I think that's easy to workaround. This is what enterprise users do; we don't really have that much of a complaint there.

We work quite closely with the local team in Malaysia — they do their job.

Generally, the initial setup has not been much of a problem. If you have some level of intermediate knowledge on networks as well as some quick training on LoadRunner, you should be able to set it up within a week or two.

Proper training is important. If you have teams that want to use the product, you need to ensure that they go through the right training. Get your guys to sit through the LoadRunner training or get someone experienced to train them.

Make sure that your team trains before they go and apply the system because LoadRunner is not actually something that you do, plug-and-play. You do need a little bit of configuration, and it's not for beginners. It is meant for people with at least an intermediate understanding of networks, and an intermediate understanding of performance application — you need to have that. I would say it's always important to ensure that you work very closely with the development team. To get the best out of the tool, you need to have a solid collaboration. When you want to troubleshoot, you want to review or uncover the performance issues; you need to make sure that you work quite closely with the development teams as well.

On a scale from one to ten, I would give LoadRunner a rating of eight. We have not used it for global distributed testing, and we also don't know its full capability from a mobile perspective. That's an area that I cannot comment on yet, so I'm reserving my judgment on that. That's the reason why I am giving it an eight.

From my perspective, there's still a gap in terms of the area that LoadRunner is being marketed to. Its biggest strength, in my opinion, is the reporting. If they could keep the reporting, but give it a lighter engine to generate virtual users, that would be perfect. 

Probably its prime advantage, it provides a centralized location for testing.

The stability on the old versions is good. On the newer versions, I think the bleeding edge is still being worked on.

It is very scalable. No issues with scalability.

Premium support is great, but before that, when we just had general support, it was not all that great. We had issues with just trying to get support to call us back on tickets and also with the turnaround time on resolution.

It is not exactly straightforward. Their instructions were not all they could have been, but we still got it installed.

The most valuable features to us are--

• Ease of uploading the scripts, and
• Script maintenance.

• Multiple users from different locations can login.
• We can view tests at run time, which has helped us to execute tests from different time zones.

Sometimes the Book Time Slot option hangs if I cancel any test or time slot. Also, if the results file gets corrupted, then we need to contact HP to get the raw results.

• Standalone PC – six years
• SaaS PC – two years

We encountered no issues when deploying.

We've had no issues with stability.

We've scaled without issue.

Customer service is excellent.

Technical support is excellent.

I have used Borland SilkTest, JMeter and LoadUI. JMeter and LoadUI have limitations in terms of the protocols. SilkTest is not as user friendly as Performance Center.

It was not complex, but I needed to consult HP support a couple of times for clarification and for help in registering the load injectors.

We did it in-house.

If the organization is going to test multiple protocols like Ajax, TruClient, Citrix, and SAP frequently, then there is definitely a high ROI in using this tool. If it's only the HTTP protocol being used, then other less expensive tools will be sufficient.

It is quite expensive.

• Auto/manual correlation
• Test Analysis
• Test script customization
• Test scheduling

Actually we’re implementing and consulting with our customers in terms of performance testing using LoadRunner or Performance Center.

For example, we’re in the middle of a project for implementing a distributed Performance Center system, with six load generators for testing an Oracle ERP system that holds about 800,000 users, with an average annual increase of 40,000 users.

Our target is to implement Performance Center, integrate it with HP ALM, design the load tests for simulating 70K virtual users that are hitting the system with different scenarios, and finally we will present reports to ERP management staff about the response trend of the system, such as virtual users passed/failed, server throughput/sec, transactions taken time etc.

I think there is a lot of configuration and customization that can come in for optimizing the recorded load test in terms of CPU and memory used and, accordingly, the service/process that are running this v-user script at load generator. This would be very helpful especially when running a huge number of v-users simultaneously.

I've used it for two years.

Running virtual users especially with a huge number is a very challenging task. The load generators may over utilize its resources (CPU/ memory) due to improper default configurations, or improper use of LoadRunner and Performance Center. Many considerations should be taken into account when optimizing resources when doing this task. For example, running the recorded script as a service at the load generator is different from running it as a process.

Also, customizing the auto-generated script in virual user generation may cause problems when ramping up the number of users. For example, simulating a generated list of variables that would be used by every virtual user is important.

Lastly, taking care of variable sizing and limitation should be reviewed, and otherwise errors may appear at certain point of running the script.

Above moderate, but HP is trying to enhance its support level.

Above moderate, but HP is trying to enhance its support level.

The initial setup is straightforward when following the installation guide steps.

I did the implementation by myself as part of a project for one of our customers. My advice is to understand the business need that would guide to a proper scope and design guidelines. Performance Center can be implemented in different ways (standalone, distributed, with/without integration with HP-ALM or HP Diagnostics, number of load generators and their location etc.). The implementor should arrange for the best and quickest setup that will fulfill the goals, otherwise the implementation would end up as a trip of trial and error, and probably fail, or exceed the time plan.

Our customers think of the following benefits as ROI of using HP Performance Center:

• Reducing downtime in operations phase that may come due to load on system by fixing or customizing their application to afford the maximum number of concurrent users they expect
• Prior knowledge of their system limitation and its break point, may let them take precautions steps at application server or network level to prohibit overload traffic that may crash their system.

The license of Performance Center is not cheap and may be very expensive for some protocols, such as SAP and Oracle. Accordingly, optimizing the license to what is only needed – in terms of protocol and number of virtual users can save a lot of money. When the load testing is limited with a start and end date plan, I recommend using temporary licenses, or a pay-as-you-go model of license, it may be more expensive in the short run, but of course it would be more economical in the long run.

From outside HP no, but inside HP there are two solutions for doing the load testing; LoadRunner, and Performance Center. The concept is almost the same, but Performance Center excels in big projects, and working with different teams besides scheduling tests feature.

My advice is to identify well the scenarios, protocols and maximum number of virtual users needed for load testing. Also, simulating a real-world load testing scenario is very important if you need to get a near-real results. For example, simulating network speed to reflect the real case. Some scenarios may require using a paid e-service, which may cost a lot when simulations repeat with a huge number of virtual users. At this point, another product called HP Service Virtualization could be used to sniff the in/out going traffic, then simulating the e-service function later after learning its function. At this point we can then replace the real e-services communications with HP Service Virtualization, so a budget-wise trade-off may be held between using HP Service Virtualization and the paid e-services. I recommend consulting HP presales to get the most proper setup with least possible licensing.

Its integration with the Application Lifecycle Management platform makes it very flexible and easy to learn.

My organization teaches Performance Center, and we find over and over that its the top choice for an enterprise solution to performance testing. Because it integrates with Application Lifecycle Management, it is a natural choice for any organization that already uses HP software solutions. While the installation of any server-based application can be complicated, Performance Center is remarkably straightforward by comparison, and for some time HP software has been known for its ease of use and, consequently, appeal to a broad user base.

Performance Center was a stand-alone server application until its marriage with Application Lifecycle Management in 12.x, when both the architecture and the user interface underwent a major overhaul. As such, it has been rapidly evolving for the past several years as HP worked through the initial UI and performance challenge in terms of a response to improve its stability, then begin to focus on adding features that would facilitate communication of performance data.

Today, they continue to enhance reporting and analysis as the My Performance Center dashboard matures, but the web-based interface is still a little clunky in terms of response to user input, especially in the Workload area where a load test is built. Communication between load testing hosts and components is very complex and sometimes fragile, making manual intervention still too often necessary should an anomaly occur during testing.

I've used it since v9.x.

As I said before, Performance Center is certainly part of a complex environment, especially now that it is an extension of Application Lifecycle Management. Yet it constantly impresses me with its relative ease of installation, and the flexibility of its architecture.

Initially (11.x) its stability left a little to be desired, but I feel HP has worked diligently to fine tune the tool so that today, the most noticeable lag occurs during the startup of a load test. Once the test is running, the tool performs as expected.

As I mentioned, I teach Performance Center classes as an independent contractor, so I do not frequently have the need to contact HP support. In my classes, naturally I hear feedback on different support experiences, and I would say they are an unsurprising mixture of favorable and unfavorable opinions. I can only imagine how challenging it must be to troubleshoot such a complex installation when every enterprise surely has a unique set of variables.

Other than HP LoadRunner, I have not. They were the first tools I learned, and have continued to use them because they constantly meet or exceed my expectations.

Performance Center 12.x is compatible with a very specific set of operating system, database, hardware, and file system options. The installation process scans for those exact options, and will not proceed even if a slight variation is detected. By comparison, the previous 11.x could be configured to use Oracle 11g (the free, development version of the database), useful when testing a trial version of the software. Now, the supported options are a bit more rigid, but still the installation process is remarkably straightforward. I have a reasonable technical background, and after working through a couple specific challenges I have now installed and configured the application half a dozen times.

We did implement the software using our in-house team, but I would advise engaging HP consultants for larger installations. The complexity of our environment cannot be compared to the needs of a typical enterprise.

The HP sales team is much more qualified to discuss topics related to pricing and licensing. HP is very focused on building long-lasting relationships with its clients, and as such is willing to negotiate a combination that is favorable for both them and the client.

HP has very masterfully re-engineered this tool and integrated it with Application Lifecycle Management to create one of the most seamless project management suites I've used in years. When simultaneously integrated with Unified Functional Testing, which also supports service testing, I feel the richness of testing information that can be managed from within a single tool (ALM) is hard to beat--despite the price tag.

The solution is a very user-friendly tool, especially when you compare it to a competitor like BlazeMeter.

The custom meter is nice. It has a lot of features.

When compared with BlazeMeter, I use the plain data. In the cloud after one year it has been very good.

With reporting, we will see the door unlock on the main portal very quickly, because LoadRunner has very good analysis tools. You can analyze data and get the error data as well. You can merge them together and dig down into specific points of time. It's great for correlating drafts within the number of users, between accounts, and with support. These functionalities are not there in BlazeMeter.

The solution is a very expensive tool when compared with other tools.

The stability in some of the latest versions has not been ideal. They need to work to fix it so that it becomes reliably stable again.

The cloud solution of LoadRunner is not user-friendly when compared to BlazeMeter. They need to improve their cloud offering in order to compete. It also shouldn't be a standalone tool.

I've been using the solution for about one year now.

In terms of stability, it depends on what you are using. Sometimes version 5.3 and the newer versions are not stable. The latest versions we are finding are not so stable when compared with the previous versions we've used, so some glitches are there. They need to rectify that. It was stable for two years, and now it's not.

The solution is scalable, and it's based on the number of licenses you have. In comparison, with BlazeMeter, I ran thousands of users, because it's very cheap and we could scale up the number of users easily with very little overhead.

In my experience, I've used BlazeMeter to scale up to 5,000 users. With OpenText, there are not more than 2,000 users.

In OpenText, I have worked with various types of clients. Some clients have platinum customer status, and some have gold. For those levels, the support will be there. At platinum, technical support is very helpful at updating their support and the support is good.

I also use BlazeMeter.

With LoadRunner, I use it with a paid tool, and since I am following the protocol, I need it to be easy to use. Whereas with BlazeMeter, we use it with JMeter. We need to use it sometimes if we want support. We need to configure some properties or some customers' ratings before we can use it.

The solution needs to reduce licensing costs. Its main competition, for example, is free to use, so I'm sure it's rather difficult to compete with it on a cost level.

We're partners with OpenText.

I haven't found many products in this particular niche that have compared to JMeter and BlazeMeter tools.

I'd rate the solution eight out of ten.

I suggest other potential users review OpenText. If the client has the budget for the solution, I'd recommend it. If they don't have a budget, I'd suggest they instead opt to look a freeware solution, and I'd suggest they evaluate JMeter or BlazeMeter.