Tricentis NeoLoad Reviews

Wasted time in low-end work (scripting) is greatly reduced with NeoLoad. It enables my team to record and modify test scripts very quickly. This reduces the effort for testing and enables the team to do more high-value work like analysis of results. We use external data analytics tools (Tableau) for result analysis. The export raw data feature in NeoLoad provides the extension with Tableau.

Performance testing is done by testing frameworks in which NeoLoad is the driver of load. The ability to extend the tool is crucial for organisations. Our test framework is composed out of NeoLoad as a centralised component surrounded by Tableau for analytics and AppDynamics for deep-dive diagnostics. This load test framework enables the team to more quickly test, find and resolve issues and system bottlenecks.

The weakest feature of most of the load test tools is result analysis. It is important we can customise graphs to visualise results in such a way that stakeholders can understand the outcome of testing. To be able to visualise results better, we extended NeoLoad with Tableau Software.

With Tableau, you can centralize performance related data in a database. Data can come from any tool, from log files or RUM tools and system utilities. Tableau provides you with the ability to connect to any data source, analyse patters and find bottlenecks more easily.  NeoLoad has made huge improvements over the last few years and finally has the ability to view the raw data (and not only aggregated results). This is fundamental for any type of performance engineering.  Aggregated results hide bottlenecks. 

Customer support is very good.

The price of the tool is competitive.

The features introduced in the recently released new version of NeoLoad fit really well into the DevOps way of working, further reducing the time for rescripting. This enables rapid delivery and lean ways of working.

The ability to test mobile applications, support cloud based testing, real-time monitoring during a test, advanced auto-correlation feature, infrastructure monitoring, detailed results reporting etc. It also gives the ability to manipulate the script by inserting JavaScript. It's one of the features that very few other tools have.

First and foremost, it took out the painful resource hours out that went in to manually correlate each and every action of the script while using LoadRunner. NeoLoad does not require much of scripting efforts, and correlation and parameterization are fairly easy.

We would like the product to be able to handle large project suits with large number of test scripts. Currently, we are not able to upload/publish huge projects back and forth from the team server to the NeoLoad controller. There is no support for VT3270 terminal emulation to support mainframe-hosted applications. There is no summary graph that shows the distribution of all of the response times on a single graph. We currently export the data as a coma separated variable (CSV) file and import into Excel to generate this summary graph.

4 years.

A small number of times, when load testing huge projects with huge number (~10,000) of vUsers, we have seen that it completely froze. We had to recover from the previously saved version of the project.

Very good, the vendor has been very accommodating with the ebb and flow of our business needs. Neotys is a partner.

Good. (They decommissioned the support for 4.2 though)

LoadRunner. Cost effective and with almost same features that are required for our project.

Straightforward and the technical support folks will be available to help.

We utilized a vendor team. They were fairly knowledgeable and knew how to quickly troubleshoot the issues or contact Neotys.

A great deal of improvement in application performance happened because we could pin point the bottlenecks. The real time performance monitoring allowed to find out the troublesome queries and tune them. Increased resource efficiency.

Low in cost when compared to tools like HP LoadRunner. Talking about licensing, they offer a cool feature that enables us to split the Vuser bundle across multiple machines. In other words, we can have multiple controllers running multiple load tests at the same time.

JMeter. Figured that it would not be suitable tool for our project and requirements.

NeoLoad is the performance testing solution for Web and Mobile applications that can also be used to run the tests from cloud by having load generator’s from various locations. If there is a strong performance team that can analyze and correlate the test results with the raw data, then NeoLoad would be the right solution.

We are primarily testing APIs but also testing the UI in some cases. In my last job, I also developed a plugin to enable NeoLoad to measure the end-user experience. If you have a modern web application, you usually only exchange JSON files between the client and the server. On the client side, you will render the JSON information to an HTML side or a dynamic DOM structure.

NeoLoad is used for testing the response time, which is not the same as the time it takes for the user to see the page. When the JSON document arrives on the client side, the client has to render the information. If you are considering the user experience from the end user's perspective, you are not interested in how long it takes to get the JSON. You want to know how long it will take to get the JSON and see the information on the page.

NeoLoad can't do this because it does not render the data from the server. The browser engine uses considerable RAM, so we cannot simulate many users simultaneously. We enabled NeoLoad to control a typical test. For example, a QF Test like Robot Framework tests the web UI. It starts and renders the application to measure how long it takes to see some special information on the page. 

These response times are fed into NeoLoad through a data exchange API, enabling us to visualize the actual user experience inside NeoLoad. That's NeoLoad's primary advantage. It has incredible APIs to implement special additional features or plugins you need for special applications.

NeoLoad helped us to transition from load testing to continuous performance measurement. It is integrated into our CI/CD pipeline. We implemented performance testing like you would have unit or functional tests. Every day, we get information if something happens in our application that affects the performance in a negative way.

NeoLoad offers better reporting than most competing tools. It is effortless to analyze and measure the reported data. It's also simple to generate a report that most people can read and management can understand. NeoLoad helps you figure out the main issues inside the application. 

Some users may find NeoLoad too technical, while other users may prefer a scripting language instead of a UI with figures and forms they have to fill in.

I began using NeoLoad when Neotys still owned the product. In total, I have about 13 years of experience working with the solution. 

Setting up NeoLoad is straightforward.

We deployed NeoLoad in-house. I already had experience with the solution from my previous job, so we didn't need Tricentis to send a consultant to help us. However, it's helpful to take advantage of their consultants if you need them. Tricentis has a network of freelancers, so it's easy to find a consultant near you, and you can usually find somebody who speaks your mother tongue.

The support for HTTP, JMS, REST API, and XBAP protocols is good. NeoLoad is very user friendly and works well with HTTP and REST AO’s. It's very effective during cloud based testing and the cost is a fraction of LoadRunner. v5.1.4 introduced the JMS testing protocol which is a copy of SOAP UI. I was pleasantly surprised that with a little tweaking and support from Neotys, NeoLoad worked with XBAP applications very efficiently.

It comes with new features, notably JMS which enables us to be on top of our performance testing tool list.

The documentation for the new JMS protocol is in its initial stages and Neotys needs to update their online support documentation with more information about it.

I've been using it for over three years.

We've not experienced any deployment issues.

There were a couple of times during a load run where NeoLoad crashed out but they were due to issues with our virtual machines rather than NeoLoad.

9/10

9/10 - the Neotys support team is superb and very efficient.

I have used LoadRunner for web, web services and Siebel protocols, but running and licensing costs were very high.

It's very straightforward.

Executable binaries made life easier during the implementation and do not require any advice.

Compared to LoadRunner, NeoLoad licensing of users is very cost effective. The cost is very simple and well defined and you can check it on the Neotys site.

You should attend a three day training course to fully understand the functioanlity of NeoLoad.

Screenshots:

It makes it easy to create load tests. We used HP LoadRunner previously, and this is far better.

It's provided a faster turnaround on load-testing creation and the ability to use the team servers so multiple people can use this at once, which we couldn't do before.

It would be nice if this could be cheaper.

I've used it for one-and-a-half years.

It's been 100% stable, which we are very happy about.

Some of the additional modules can be pricey, but it is scalable.

Compared to support at our previous product, they're more responsive and they're more knowledgeable. Overall, it's three times better than our previous solution.

The initial setup was very straightforward.

We implemented it in-house.

We looked at JMeter, but NeoLoad had what we needed. We were using LoadRunner for six years and then we replaced it with NeoLoad.

Everyone's needs are a little bit different, so make sure you do a good POC.

The ability to quickly record and correlate a script very quickly with a lower learning curve. A complex LoadRunner script could take days to create, but the same script in NeoLoad could be created within hours. We had contractors coming in, and within months they were able to script and start contributing to the team very quickly due to the simplicity of this software.

NeoLoad has the ability to compare a test report to a previous run. We can easily compare to see if this new code base is performing better or on par with the previous code base.

We were able to create more scripts using NeoLoad within a short time frame. We increased the number of applications we support/test due do the quick scripting time. Now we have more time to do analysis and engineering work which is very important to us.

The ability to show transactions per second during the test run. Currently, we have to eyeball the TPS using the graph. NeoLoad 5.2 will have TPS reading after the test is completed but not during the test run.

16 months

None.

None.

None.

9/10

We used LoadRunner before we switched over to NeoLoad. LoadRunner is a powerful tool, but the learning curve is very high. It would not be feasible for our team because we have a lot of newcomers and a lot of applications to support. NeoLoad costs a fraction of what Load Runner costs.

Setup is very straightforward. Neotys also provides guidance and support, setting up the team servers and licenses.

We did the installation in-house, with vendor support when needed.

Identify what kind of testing your company needs (mobile, database, web, and so on) and find out if NeoLoad supports it.

1. The GUI based scripting. Other tools are more script/code oriented. This is a huge time saver for us since we don’t have a dedicated performance team (yet). I can create scripts quickly and get tests going.
2. The comparison report feature is good for pre/post-change result comparisons.
3. The ability to write custom JavaScript actions. This has allowed us to handle JSON posts in web requests, which has been great.
4. LAN/WAN emulator has been great in being able to simulate remote locations in their network connections.
5. It ties into Dynatrace out of the box which we’ve used for deep-dive application performance testing as well as defining business transaction SLAs.
6. The ability to create custom Frameworks for correlation values of custom, in-house apps.

It’s allowed us the ability to look at the performance of an application prior to it going to production utilizing an automated tool. Up to this point, most, if not all, of it was done manually. Projects are now making time for performance testing into their timelines.

Some of the documentation is geared more towards Java. We are an all-Windows shop and it would be great to see some C# examples on how to integrate a custom performance counter from functional testing tools into NeoLoad.

It's been almost two years since we began using it.

We didn't encounter any issues with deployment.

We haven't had any issues with stability up to this point.

You can deploy as many LGs as you have a license for.

8/10 - Our account rep, Evan, and engineer, Steve, were great to work with when we were looking at the product. Ever since, they’ve been responsive and helpful.

10/10 - The engineers know their product and know performance engineering.

Prior to NeoLoad, we weren’t using any automated tool for performance testing.

The initial setup was straightforward. The setup of an LG or Controller is as easy as a selection during the install. It’s up to the user how complex they want their environment to be.

We implemented it in-house ourselves. Evan and Steve provided guidance while doing so.

I wouldn’t know the monetary side, but the ability to handle SignalR requests by dragging a ‘fork’ action onto the script speaks for itself. No extra coding or anything is needed.

To take it a step further, NeoLoad is smart enough, post recording, to identify and account for the presence of something like SignalR, and will create a copy of the script with the ‘fork’ action already present.

At the time of purchase, it was a competition between Neotys and HP. Neotys’ licensing model was also different from HP, although HP was transitioning into a similar model as Neotys while we were going through the process.

We evaluated HP LoadRunner. I’ve used it before and it is/was the industry standard tool.

If you’re coming from a code-based solution, spend time with Actions to become familiar with them. Additionally, if you’re using an APM tool like Dynatrace, test out the integration just to see what it gives you.

In my opinion, correlation of dynamic data is the most important advantage of this tool. Normally, it is difficult to record performance scripts, but with this tool it is much easier.

Since this tool is lowering the risk of monthly releases to live environment, it reduces the stress on people. In addition, performance problems that we could not solve for a long time have been solved by the tool with Dynatrace integration.

It would be good to make some updates on the reporting side. For example, correlating the performance of server, throughput and hit count in a specific time interval would be a nice functionality.

We use the tool full-time on our test environment and we have been releasing it to our live environment for 2 years.

Support side of the tool is very strong. We have seen fast feedback and answers to other questions we faced in the past about licensing and other general problems.

I have used many performance testing tools. This tool goes one step further with easy integration with code analysis tools, server monitoring, the success of correlation studio and the advantages on the mobile side.

Licence cost is very attractive compared to other vendor tools and also there are many license alternatives. Licence costs are good enough compared to the benefit of the tool to the company.

The ease of recording scripts and the correlation process, the force-on monitoring, success on the mobile side, and easy integration with code analysis tools are the reasons I have rated this product so high.