OpenText Enterprise Performance Engineering (LoadRunner Enterprise) Reviews

We use OpenText LoadRunner Enterprise for load testing. 

The tool is very easy to set up and get running. 

OpenText LoadRunner Enterprise needs to improve reporting. 

I have been using the solution for 13 years. 

I rate OpenText LoadRunner Enterprise's stability a nine out of ten. 

I rate the tool's scalability an eight out of ten. My company has 120 users. 

I rate the tool's deployment a nine out of ten. It can be completed in two to three days. 

I rate the product's pricing a three out of ten. 

I rate OpenText LoadRunner Enterprise a ten out of ten. 

Performance of load test applications, reliably, and with good reporting.

We have high confidence that the results are reliable. It's helped catch multiple bugs before they've gone to production. And we're able to more confidently release software into production, just more confidently release code.

I'd like to see better reporting, ultimately, and analysis. I'd like to see the analysis more accurately pin-point where these issues are; a little more feedback immediately in terms of the anomalies that we're seeing during the testing, so we can get alerted much quicker, instead of blowing a whole bunch of our day to test data during a test.

The reporting could be a lot better. It's cumbersome. The VuGen takes a long time to load up, and edit and execute things off those. So a lot of the basic infrastructure.

We've had a few crashes, but overall they seem to be more of the aging architecture, the hardware's not as reliable. A few software bugs in there, which we have to work around. But overall, it's been pretty reliable when you think about how much load goes through it and the number of people using it.

We've scaled it up to, well, to our limits at least. A good 1500 users and some 10 million transactions, or something like, that per hour. It's generally handled that pretty well. We're looking forward to moving into StormRunner next.

The tech support seems to be pretty good. There have been a few problems that they haven't resolved, but they've led me in the right direction to eventually figure it out on my own. It's very difficult for them to understand all the wide range of applications we use. So I'm really impressed with how ultimately knowledgeable they are, given that they don't actually use all these applications that are out there.

I wasn't involved in the initial setup but I've been involved in all the upgrades. They've done it multiple times. Each upgrade seems to be adding a lot more features. It seems like the pace of development has picked up in the last couple years especially, and we've been impressed with the direction that the product's going.

The upgrades actually were fairly complex for my team. The last one, they were actually giving up for a while. It turned out that there were some database versions that were incompatible. Eventually, once we were able to get connected with the support, we were able to get most of it worked out.

There was a big upgrade with the 12.5 version that was a little slower. But everything else has been very smooth since then. Still, it's difficult. Everybody's come to count on that software being at a certain version.

The most important criteria when selecting a vendor are, I suppose, support, and the reliability in the industry.

They have a good road map for moving forward. It's going to be integrating with a lot more other products that the DevOps world is starting to push upon us.

It's not quite perfect. It's still the best in the market. It holds up to a lot of the scrutiny that the developers constantly throw at us.

There have been a few issues, especially around the scripting and the IDEs, handling of some of the protocols, that still don't perfectly match with what we have in production. But overall, we're able to pretty well defend it. You can almost always reliably pull up the application, and the response times seem to match what we have coming out of LoadRunner.

I would definitely advise looking at Performance Center, it's still the best in the breed. Just make sure that you have a good team in place that can implement it.

The fact that you can have tens of thousands of virtual users and just expand an army of load generators to hammer on whatever application you're testing. It seems to be so scalable, and we haven't approached any limits. We have some projects that have over 100 load generators and they don't have many issues using the tool.

I think better support for cloud-based load generators would help. For example, integrate with Amazon AWS so you can quickly spin up a load generator in the cloud, use it, spin it down. That way, you're only paying for the actual time that it's being used. I know they have some functionality with that right now, but it could be improved because right now, our projects have to bare the cost of that infrastructure, whether it's an AWS or whether it's their own VMs, so that would help out with that.

Personally, I've been using LoadRunner or Performance Center for almost 10 years. But our deployed solution, that's available to projects, has only been deployed for about two and a half years.

I think Performance Center is pretty stable. It's the enterprise version of LoadRunner, which was a very mature tool. In fact, a lot of the components or installation files are still the exactly same. They've added this central, enterprise web front end to it that works pretty well. It's compatible on multiple browsers, on ALM. I would say it's a pretty mature, stable product.

More people seem to know how to use Performance Center, so we don't need as much help with it, but it is a more complicated product. HPE, or Micro Focus, has been responsive.

This is complex. We are hosting some of the components in the Deloitte network. The clients or the projects have to set up their own load generators. You have to configure firewall rules. We have to install these agents and point them at our environment to connect and troubleshoot connectivity issues. And every client has a different need. If you're behind a VPN - the app that you're testing - or it's inside the client network, that introduces some challenges. Just by the nature of the tool and what you're doing, it is complex.

It does everything you could hope for in a performance testing solution. It's not cheap, but that's not necessarily a concern for us because we're a large company. But it does anything you can think of. It's a pretty mature, robust tool.

• Auto/manual correlation
• Test Analysis
• Test script customization
• Test scheduling

Actually we’re implementing and consulting with our customers in terms of performance testing using LoadRunner or Performance Center.

For example, we’re in the middle of a project for implementing a distributed Performance Center system, with six load generators for testing an Oracle ERP system that holds about 800,000 users, with an average annual increase of 40,000 users.

Our target is to implement Performance Center, integrate it with HP ALM, design the load tests for simulating 70K virtual users that are hitting the system with different scenarios, and finally we will present reports to ERP management staff about the response trend of the system, such as virtual users passed/failed, server throughput/sec, transactions taken time etc.

I think there is a lot of configuration and customization that can come in for optimizing the recorded load test in terms of CPU and memory used and, accordingly, the service/process that are running this v-user script at load generator. This would be very helpful especially when running a huge number of v-users simultaneously.

I've used it for two years.

Running virtual users especially with a huge number is a very challenging task. The load generators may over utilize its resources (CPU/ memory) due to improper default configurations, or improper use of LoadRunner and Performance Center. Many considerations should be taken into account when optimizing resources when doing this task. For example, running the recorded script as a service at the load generator is different from running it as a process.

Also, customizing the auto-generated script in virual user generation may cause problems when ramping up the number of users. For example, simulating a generated list of variables that would be used by every virtual user is important.

Lastly, taking care of variable sizing and limitation should be reviewed, and otherwise errors may appear at certain point of running the script.

Above moderate, but HP is trying to enhance its support level.

Above moderate, but HP is trying to enhance its support level.

The initial setup is straightforward when following the installation guide steps.

I did the implementation by myself as part of a project for one of our customers. My advice is to understand the business need that would guide to a proper scope and design guidelines. Performance Center can be implemented in different ways (standalone, distributed, with/without integration with HP-ALM or HP Diagnostics, number of load generators and their location etc.). The implementor should arrange for the best and quickest setup that will fulfill the goals, otherwise the implementation would end up as a trip of trial and error, and probably fail, or exceed the time plan.

Our customers think of the following benefits as ROI of using HP Performance Center:

• Reducing downtime in operations phase that may come due to load on system by fixing or customizing their application to afford the maximum number of concurrent users they expect
• Prior knowledge of their system limitation and its break point, may let them take precautions steps at application server or network level to prohibit overload traffic that may crash their system.

The license of Performance Center is not cheap and may be very expensive for some protocols, such as SAP and Oracle. Accordingly, optimizing the license to what is only needed – in terms of protocol and number of virtual users can save a lot of money. When the load testing is limited with a start and end date plan, I recommend using temporary licenses, or a pay-as-you-go model of license, it may be more expensive in the short run, but of course it would be more economical in the long run.

From outside HP no, but inside HP there are two solutions for doing the load testing; LoadRunner, and Performance Center. The concept is almost the same, but Performance Center excels in big projects, and working with different teams besides scheduling tests feature.

My advice is to identify well the scenarios, protocols and maximum number of virtual users needed for load testing. Also, simulating a real-world load testing scenario is very important if you need to get a near-real results. For example, simulating network speed to reflect the real case. Some scenarios may require using a paid e-service, which may cost a lot when simulations repeat with a huge number of virtual users. At this point, another product called HP Service Virtualization could be used to sniff the in/out going traffic, then simulating the e-service function later after learning its function. At this point we can then replace the real e-services communications with HP Service Virtualization, so a budget-wise trade-off may be held between using HP Service Virtualization and the paid e-services. I recommend consulting HP presales to get the most proper setup with least possible licensing.

Its integration with the Application Lifecycle Management platform makes it very flexible and easy to learn.

My organization teaches Performance Center, and we find over and over that its the top choice for an enterprise solution to performance testing. Because it integrates with Application Lifecycle Management, it is a natural choice for any organization that already uses HP software solutions. While the installation of any server-based application can be complicated, Performance Center is remarkably straightforward by comparison, and for some time HP software has been known for its ease of use and, consequently, appeal to a broad user base.

Performance Center was a stand-alone server application until its marriage with Application Lifecycle Management in 12.x, when both the architecture and the user interface underwent a major overhaul. As such, it has been rapidly evolving for the past several years as HP worked through the initial UI and performance challenge in terms of a response to improve its stability, then begin to focus on adding features that would facilitate communication of performance data.

Today, they continue to enhance reporting and analysis as the My Performance Center dashboard matures, but the web-based interface is still a little clunky in terms of response to user input, especially in the Workload area where a load test is built. Communication between load testing hosts and components is very complex and sometimes fragile, making manual intervention still too often necessary should an anomaly occur during testing.

I've used it since v9.x.

As I said before, Performance Center is certainly part of a complex environment, especially now that it is an extension of Application Lifecycle Management. Yet it constantly impresses me with its relative ease of installation, and the flexibility of its architecture.

Initially (11.x) its stability left a little to be desired, but I feel HP has worked diligently to fine tune the tool so that today, the most noticeable lag occurs during the startup of a load test. Once the test is running, the tool performs as expected.

As I mentioned, I teach Performance Center classes as an independent contractor, so I do not frequently have the need to contact HP support. In my classes, naturally I hear feedback on different support experiences, and I would say they are an unsurprising mixture of favorable and unfavorable opinions. I can only imagine how challenging it must be to troubleshoot such a complex installation when every enterprise surely has a unique set of variables.

Other than HP LoadRunner, I have not. They were the first tools I learned, and have continued to use them because they constantly meet or exceed my expectations.

Performance Center 12.x is compatible with a very specific set of operating system, database, hardware, and file system options. The installation process scans for those exact options, and will not proceed even if a slight variation is detected. By comparison, the previous 11.x could be configured to use Oracle 11g (the free, development version of the database), useful when testing a trial version of the software. Now, the supported options are a bit more rigid, but still the installation process is remarkably straightforward. I have a reasonable technical background, and after working through a couple specific challenges I have now installed and configured the application half a dozen times.

We did implement the software using our in-house team, but I would advise engaging HP consultants for larger installations. The complexity of our environment cannot be compared to the needs of a typical enterprise.

The HP sales team is much more qualified to discuss topics related to pricing and licensing. HP is very focused on building long-lasting relationships with its clients, and as such is willing to negotiate a combination that is favorable for both them and the client.

HP has very masterfully re-engineered this tool and integrated it with Application Lifecycle Management to create one of the most seamless project management suites I've used in years. When simultaneously integrated with Unified Functional Testing, which also supports service testing, I feel the richness of testing information that can be managed from within a single tool (ALM) is hard to beat--despite the price tag.

• It helps you to build custom design application simulation models that enable the performance test engineer to assess an enterprise, multi-tier, heavily-distributed, high-traffic application on how the application for that particular release meets its non-functional test requirements and SLA thresholds, which are key business-transaction response times.
• It tell you your applications' CPU computational power usages under various conditions such as stress, volume, and scalability.
• Its distributed nature will let you choose load generators that can be geographically located anywhere, thus traffic simulations from multiple locations make it possible to create a real production scenario.
• The true advantage of Performance Center is the ability to enable multiple engineers to do design and validate their script locally, but to execute them globally by using global resources spread across an organization.
• The other advantage is the integration with HP SiteScope which lets you to perform deep-dive monitoring of your application under test during your test execution for live health analysis using several readily available monitoring templates.

Some of the key-business and mission-critical applications released earlier into production were having issues in terms of key business transactions running slowly, which were impacting the end-user experience. The slow business transactions were causing grief to several customers, which eventually led to customers being moved onto different competitor products, causing revenue loss. There was increased downtime of applications in production due to poor application performance.

With the help of HP Performance Center and LoadRunner, we were able to critically evaluate the applications performance and stability, and were assured that each application that was released into production was performing better and meeting all SLAs, including transactional response times and system and platform resource utilization.

We have received excellent feedback in one specific instance where a customer came directly to me, and said that they are seeing much better transactional response times, which helped them serve customers faster, and now they are seeing their customers coming back.

Integration with other APM tools like DynaTrace or AppDynamics is a little tedious. Plus, support around cloud solutions and architectures needed to be improved.

I have used several versions of this product for over five years, alongside HP LoadRunner.

No issues encountered.

No issues encountered.

No issues encountered.

It's excellent.

It's excellent.

As a person who has been in the performance engineering field for several years, I have used several similar products. However, Performance Center and LoadRunner offer unbeatable support across different protocols, including SAP and AJAX true client.

Also, its ease of use in designing and reusable custom automated performance frameworks is unbeatable. Its support in designing frameworks and scripts for load testing message queues, web, and web-service protocols are quite remarkable.

It offers different types of users for people who come from different programming backgrounds i.e. if you are predominantly a C programmer, you can write a lot of custom API’s using C, and similarly, if you come from Java programming you can use your Java skills in custom API implementation.

It was straightforward.

In-house.

It can lead to the generation of a very high ROI if you have the right people with the right expertise of the tool set.

It is a bit highly priced. However, better products with a better quality can come with a good price.

It is definitely one of the best products available on the market. Definitely programming knowledge around C programming would be greatly advantageous.

The primary use case for clients is that they often have large application development teams and application development projects that they needed to scale. So, for instance, if they were developing a new banking website and they needed to check that the application that sits on that website was scalable from a few hundred concurrent users to many thousands of users, they could test the load response using LoadRunner.  

That is what LoadRunner does, it does the performance testing and measures load-bearing response.  

I think, for me, the most valuable part of the product is the way you can do the basic testing. You can create the test script and then simulate thousands of users very, very easily. Instead of having to have lots-and-lots of systems that would emulate users, you just needed a couple to emulate tens of thousands of users. So the scalability of LoadRunner while it was testing scalability is really valuable.  

To improve the product, I think the integrations could be a little bit more slick. It does handle a lot of great integrations, but then some of them can be a little bit clunky to implement. The integration with third-party tools needs to be stepped up a little bit.  

As far as other things that need to be added, it has changed quite a lot recently, and I have not had vast amounts of experience with the latest version. So I am afraid it would not be fair for me to go further in expounding on that question. Things that I talk about may already have been included.  

I was a presales consultant and so I was kind of a technical consultant as well. I was working with the solution end-to-end for about seven years. My main focus was not LoadRunner for the entire time, but I gained knowledge of LoadRunner and I gave presentations about it. It has been about eight to twelve months since I last did anything serious with it. However, I am still familiar with the product.  

The scalability is really very good. In some ways, it is the purpose of the product: testing by use of scaling loads.  

The initial setup is pretty straightforward. But I have got to say, having worked on-and-off with LoadRunner over a period of time, I knew kind of instinctively how to set it up after a while. In other words, in my case, I would say it was simple. On the other hand, I think it the first time I tried to set it up it was a nightmare. After that, it was easy because I learned a lot about it. If I had to score it out of ten for initial setup with ten being the best, it would probably be seven-out-of-ten. It is not really going to be super easy for first-timers to deploy.  

The deployment could take quite a while, even when I was used to doing it. Getting the software installed and running is pretty quick and that is not a problem. But creating the projects and creating the test scripts can take a little while. To get up and running and doing stuff within it, it is probably just around a week. Doing it professionally with the integrations and with all the correct testing scripts, it can take a month and more. It really all depends on the purpose and how you want to use it.  

My advice to people considering LoadRunner is that if you are going to use the product, use it as part of your everyday application development lifecycle. Do not just use it right at the end, because it gives you some great insights during the development phase as well as at the end. You will end up writing cleaner application code with it. So bake the use of LoadRunner into your full application life cycle.  

On a scale from one to ten where one is the worst and ten is the best, I would rate this solution as between and eight and nine-out-of-ten. I could be slightly biased, having worked for the company that sells it. But it is a very good, professional solution. With the latest updates, it is very comprehensive and one of the best products of the sort. Let's say nine-out-of-ten because there is always room for improvement.  

As a tool, it's something that we can use in all aspects of performance testing, whether it's services, whether it's web, whether it's customized APIs, like Citrix. It's the tool for performance testing, and it's definitely the industry leader that I've been using for years. It's a very powerful tool.

Truthfully, I think performance is a fairly mature space now. There are not too many things popping up that we're saying, "You know, Performance Center doesn't do X, Y or Z." It's a pretty mature tool and we're happy with it.

In terms of having one tool it is helpful in terms of training. It has really great reporting features, so not only is it a good tool to do testing with, but when it comes to helping you produce good results that you can present up, it's an all around tool that has everything that you need within it. You don't have to go to other third-party tools for reporting or for different types of testing. It's a tool that is "one size fits all".

I think better or more integration with some of the monitoring tools that we're considering.

We're looking to bring in maybe AppDynamics. I personally don't know the integrations but having good integration tools is going to help us in the future.

I think some of the reporting features could be better. I haven't seen much change in terms of that aspect of it, the report analysis piece. It's been good enough but I haven't seen as much advancement in that space, the reporting analysis.

I find it very stable. I will say that with the changes to virtual machines and things like that, It is harder now to manage. That's one of the reasons we are looking at not only using Performance Center, but also StormRunner. That gets us away from needing to scale up; doing that within Performance Center can be difficult because you have to deal with the aspects of all that infrastructure. It's not the tool itself, but it's the underlying infrastructure that you have to manage. Something like StormRunner is promising because it gets you away from that a little bit. Somebody else is taking care of that aspect of it.

It's an enterprise type tool. It scales very well. Again, it's not the product that we have issue with scaling.

We have challenges with the number of VMs that we have. The issue with scaling up is we come up against our infrastructure team that wants to limit the number of VMs. They have to manage them. The whole VM-type design seems to be a lot more VMs, and when you need to scale up, it's even more VMs. I understand the cost savings with VMs but in terms of management, if you don't have that nailed down in terms of automation and all that monitoring, it's a challenge to scale up.

I think that's where our current Performance Center implementation is probably going to remain, but if we're going to scale up, we're really looking toward something with StormRunner, where we can scale up as we need and not have to worry about the whole managing of the infrastructure, cause that's a challenge.

I have used tech support. Not recently, not in the last 12 or 18 months, but yes, I've used tech support. They're responsive. I've had good support. They get to the point. There's not a lot of hand-holding, they expect you to know what you're doing. I have no problem with that. As long as I can get the answer, get what I need and get it done, I'm happy with that.

I wasn't involved in the initial setup, it was pre-existing. We use Performance Center, but as the HPE ALM piece is managed by a separate tools group, that's a challenge because we don't have control over the whole implementation.

I've assisted with the Performance Center upgrade piece a little bit, installing it, but in general, we have a whole separate group that does it.

When looking for a vendor to work with the number one thing is does a tool do what it needs to do? Second, of course, support. Stability and the ability to scale are pretty important but I think that's grouped under the tool itself. It has to be an enterprise ready scalable tool.

Regarding the vendor itself, support, being responsive, having a way to access the support that's not overly obtrusive. I don't mind doing emails or logging onto a website, just as long as it's not too convoluted. Sometimes I feel like you have to go through 20 steps to get somebody to call you back and every customer support or technical support has their process. As long as it's not overly going through hoops to be able to access that.

In terms of advice, you have to do the math. There are a lot of free tools or tools that you write yourself. You just have to make sure that, long term, are those things maintainable, supportable? Do you have the training? Do you have the support? You have to bake all that in before you make a decision. It's not to say those other tools aren't valid, and people do a lot with them but, for example, if the tool needs programming skills, do you have those skills? Do you have a team with those skills? And how much is it going to cost to keep, hire, or maintain your staff with those tools? So you have to do the math to make that kind of decision, what the right tool is.

I think the tool does what it really needs to do and I've never had an issue with their support. I think they're definitely the industry leading product for performance.