Tricentis NeoLoad Reviews

We had three specific use cases we targeted Tricentis NeoLoad for. The first was for services, the second for terminal emulation mainframe applications, and the third was for the web.  We are looking for a more cost effective, efficient (low code) alternative to our current framework. 

Tricentis provides great documentation and collaboration.

The most useful aspect of Tricentis NeoLoad was the low/no code for the web.

Tricentis NeoLoad could improve the range of codeless scripting such as with terminal emulation apps. 

We did a trial of Tricentis NeoLoad for approximately three months.

Tricentis NeoLoad is a stable solution.

We did not have any problems with the scalability of Tricentis NeoLoad but did not fully push the product.  We have approximately six or seven people involved in the POC. This included senior engineers and test developers.

We do not plan to expand the use of this solution at this time because it does not meet all of our needs.  Would consider a second look in the future.

The technical support from Tricentis NeoLoad is very helpful.

We have used Micro Focus LoadRunner. However, it is expensive and there appears to be a declining number of users.  Tricentis NeoLoad is more affordable and would better enable our pipeline but has fewer features.  We are not in a hurry to change but will continue to review products of interest and will continue monitoring Tricentis product roadmap.

The initial setup of Tricentis NeoLoad was straightforward. We had the installation instructions and the packages, and it took us approximately four hours to complete.

We did the implementation in-house.

Tricentis NeoLoad price is a benefit of using this tool, it is less expensive than some of the other solutions.

I would encourage others to take a look at this solution. It is a good solution and they maintain the technical expertise needed for success.

I rate Tricentis NeoLoad an eight out of ten.

The two standout features for us are:

• Team Server Collaboration - as is the case with many MNCs like ours, teams/team members are spread geographically at various locations. The Team Server module is a boon for such teams as multiple users can work on the same project simultaneously and check-in the changes. It also helps manage and allocate virtual users (of shared license) for execution of multiple projects at same time.
• Results & Analysis - NeoLoad provides comprehensive results for any load test. The results can be drilled-down to minute levels (from scenario level all the way to individual request level) of any transaction which makes root cause analysis very effective. NeoLoad can also consolidate results from different sources and the results are graphical (one can build custom graphs for any transaction) and colourful for easier interpretation.

Other advantages are

1. GUI driven recording makes it easier to record and prepare test scenarios without need for extensive coding knowledge thereby enabling tool users to ramp up faster
2. Supports a ton of protocols plus additional plugins such as Jenkins, AppDynamics, etc.
3. Custom scripts and frameworks can be built
4. Recording and simulating mobile applications is simple yet realistic

NeoLoad is a robust and versatile tool. With plugins for CI tools like Jenkins, NeoLoad is configured to automatically trigger small load tests on modules each time there is a build update. This has enabled us to identify performance issues for our clients at a very early stage. It has undoubtedly an impact during larger scale enterprise performance tests when all the modules are integrated and tested as cost to fix performance issues increases with later stages of deployment.

We have raised a few support tickets for certain feature sets/improvement areas.
These include permanent fixes for a few stability issues discussed in the following question and implementation of more robust solutions for media streaming protocols, TCP/IP and websocket protocols.

Also, there are a few changes in the latest version, 5.2, which have brought in many improvements but have removed certain features. For example, v5.2 has done away with the 90th percentile reporting for response times, which was present in earlier versions. While this is a small deviation, it still is important for organizations using it.

The tool can hang and take a long time to open projects if the size is large and if many graphs and result templates are stored. Certain protocols although supported by NeoLoad need to have custom solutions built (e.g. web socket connections) while solutions for certain other protocols such as media streaming protocols like HDS, HLS, etc. are still in the nascent stage.

We have used NeoLoad since late 2009.

There is no specific maximum project size defined (actual NeoLoad project size). This size is dependent on number of user journeys in the project, number of scenarios defined, number of graph templates stored, and number of results stored and so on. The bigger the project, the slower it opens in the tool (as expected) but many times can be excruciatingly slow. The other drawback is that the graphs and results may not get saved correctly. With the latest versions of NeoLoad, this issue has been significantly fixed but it can still occur at an undesirable moment as the optimum size of a project can only be determined by trial-and-error. Our experience has taught us to keep multiple backups of same project under two categories for such exigencies – the first category would contain backup of the project with previous results and graphs while the second one would have only the project (VUs and scenarios) but with previous execution results and graph templates removed.

The other issue we faced was with respect to results generation. If we need to abort a performance test that is in progress, all one has to do is click the stop button. However, if the stop button is clicked twice inadvertently or clicked twice because it still showed “consolidating results” message for eons after the first click, the results ultimately generated would get corrupted. Either the results would not show for all virtual users executed or would not get generated at all. The last time we checked, Neotys did not have a concrete solution to this problem even after analysing the execution logs and troubleshooting it. Their “solution” was to check our stop policy for the scenarios executed (to make sure all virtual users reached their ‘end’ state to generate results) and also to check if any antivirus or firewall was impeding result generation (an antivirus or firewall can prevent NeoLoad from writing results to your folder). From our side, our stop policies were fine and there was no antivirus/firewall to block NeoLoad so the solutions were not that useful. As a workaround, we would click just once to abort an execution run and wait for the results to be generated even if it took a while.

The customer service and technical support is excellent if you are a paid customer. The turnaround time and SLA for a priority one - Urgent - issue/ticket is almost immediate. For less priority tickets it is quick (usually a couple of hours) and the representatives even help build a custom solution for your project if need be. However, if you are using the trial version, you would have to depend on NeoLoad forums for answers. On the flip side, Neotys documentation available on their side is detailed and exhaustive which should answer most questions when you are evaluating the product.

We have used a host of other solutions such as LoadRunner, JMeter, WebLOAD, SOASTA, etc. While the choice of the solution depends on various factors such as open source vs commercial tool, protocols used, number of virtual users etc. NeoLoad comes up trumps on most factors which were needed for our clients namely:

1. Cost
2. Ease of deployment and scalability to test for high user loads (in the range of 100,000s users for large enterprise tests)
3. Ability to monitor performance at various levels of architecture
4. Graphical representation of results
5. Detailed comparison can be seen in the sheet attached

Initial setup is a breeze. The installation is a complete package without having to switch between standalone modules for recording scenarios, execution and analysis which is not the case in say, LoadRunner or WebLOAD which has different modules for each (like VuGen, Controller & Analysis).

Neotys also offers their own cloud infrastructure to leverage upon for tests with larger user loads which can be configured to each project needs. However, if you are setting up your own cloud infrastructure on say, Amazon AWS, you would have to be equipped with enough knowledge to set up the images (AMIs) and configure the necessary IPs & locations for the test.

When we started out in 2009-2010, we did use the vendor team to guide us through and setup the necessary infrastructure. Over the years, we have built a strong in-house team and have setup our own infrastructure and cloud instances.

Depending on the technical expertise of your team, you can choose to implement in-house or avail the support of the vendor team. While the basic installation is straightforward, the vendor team could help you choose the right hardware systems/configuration for localized load generation or help you configure Neotys cloud for IP simulation or handling larger user loads.

Neotys offers various licensing plans to suit every business. The free version comes with 50 users with all protocols included for small load tests. The paid licensing plan depends on number of virtual users needed (to simulate the load), duration, the protocols under test, number of concurrent applications being tested, cloud usage etc. For example, the licence can be availed even for a day, a month, or a year depending on the need. In that sense, the licence cost can range from $1500 to >$20000. A general rule of thumb is the longer the licence period and the greater number of virtual users results in a lower price over the long term.

Leverage the free version of performance tools to its maximum capability to evaluate and cross check against the performance testing needs for your business to make an informed decision on buying the tool. The pros outweigh the cons and so NeoLoad is highly recommended over its competitors.

We are using it for performance testing.

It's improving our test lifecycle time. It's taking less time compared to the previous tool.

The Frameworks feature is valuable. NeoLoad Web and the API are also valuable. It provides API support.

The overall stability of the GUI should be improved. The GUI component is not stable enough. We have observed crashes several times. 

There should also be support for more protocols for desktop applications.

I have been using this solution since 2019. It has been four years. 

I'd rate it a four out of ten in terms of stability. The GUI isn't stable enough.

I'd rate it a seven out of ten in terms of scalability. 

Within our team, we have around 25 users. Their roles are testers, test leads, and test managers.

We aren't planning to increase its usage. The way we are using NeoLoad and other tools is pretty streamlined. We are using NeoLoad 90% as per our requirements, and we aren't planning to increase its usage.

I'd rate their support a four out of ten. The problem is that we have to raise a support ticket in the portal. They usually take a long time to reply. The first or second response from the team usually covers the basic tasks, which the team would have already done before reaching out to their support team. So, by the time we get a proper response from them, we might have moved away from that particular project or we might have found some alternative. That part is a bit bad.

Neutral

We were using Micro Focus LoadRunner. We switched because of the organization's requirements.

As compared to other tools, the initial setup is easy, but there is some complexity that comes with the Kubernetes-based setup of one of the components. So, it's a bit easier than others, but there are certain aspects that make it difficult. I'd rate it a six out of ten in terms of the ease of the setup.

Its deployment took a couple of weeks, which also included testing. 

It was deployed in-house. We deployed it ourselves based on the documentation.

For deployment, we only need one employee who has experience in cloud deployment. For maintenance also, one admin is enough. 

I'd rate it a seven out of ten in terms of pricing.

We evaluated open-source tools such as Scala and JMeter. We went for NeoLoad because of the collaboration feature, which isn't there in Scala and JMeter. Another reason was the enterprise support, which doesn't come with open-source tools.

It's definitely worth a try. Compared to LoadRunner and other tools, it gives more flexibility and is easier to maintain. 

Overall, I'd rate Tricentis NeoLoad a seven out of ten.

The primary use case is for web applications and also SAP GUI—not SAP Applications but the UI version of SAP. For both these areas, it was working fine from the NeoLoad side.

The solution has been moved to the cloud version.

The first benefit is that it's quite cheaper. In the market there are mainly three tools which dominate: JMeter, LoadRunner, and NeoLoad. JMeter is open source. LoadRunner is quite expensive but has quite good features. NeoLoad is competitively quite cheaper compared to LoadRunner, and it provides great features, except in some areas. For instance, it doesn't allow some SAP applications. There are some areas where it's a little bit tough but other than that, it's quite good.

I feel that the codeless part, the dynamic value capture part is quite easy in NeoLoad compared to other tools. JMeter is a little bit tough, but LoadRunner gives you many features. For any codeless value, it is easy to detect from the NeoLoad side.

The SAP area could be improved. Not the GUI applications, SAP log-on, or something like that. We can see features and use NeoLoad properly in all the normal SAP log-on areas. I haven't tried it, but I have heard that Citrix-based applications don't work properly in the SAP application. In those types of different areas, there could be more protocol flexibility.

If there is a high number of users on NeoLoad—like around 5,000 or 6,000—it sometimes feels like that might be a bottleneck for the performance. That could be improved. With more users, there shouldn't be any issues from the tool side.

With scripting in a different browser, sometimes there are compatibility issues and it doesn't generate the script properly. Those types of crashes could be removed.

I was using NeoLoad six months ago because of some performance testing issues. Previous to using NeoLoad, we used LoadRunner.

The stability is good. It's quite reliable. Compared with a different browser, it sometimes creates problems and bugs. With the number of users increasing, sometimes the CPU and system where it was connected increases because of trade concurrency or something like that. That can be a little bit problematic. Other than that, things are fine.

Ongoing maintenance is not required. Whenever we have deployed the tools, if any version upgrade happens, we just check what the upgrade is exactly and someone on the deployment team connects the documents and everything. They check the document accordingly and update the version. It's not complicated.

We have roughly eight to nine performance testers, including the performance lead, the performance engineers, and the performance testers. Everyone will be running different projects. We'll be using the tool and running the performance testing based on the license and everything. The user count license will be ramping up the load. There are instances where multiple teams are using NeoLoad at the same time with different licenses. There has been no conflict error with each other, and we haven't seen a problem with different license things. Because we have two licenses, two tests can be executed parallel to each other.

The solution is scalable. After the number of users increases to 5,000 or 6,000, we have occasionally found problems. Below that, things are fine and the response times are good.

Tech support is okay. When we had 50 users, we reached out to the tech support team and within two to three days we received a response. The time took longer because it was a complicated problem. Response time is perfect.

I previously used LoadRunner. One of the reasons we had to switch to Neotys was because it was quite cheaper, but I still prefer LoadRunner to NeoLoad.

The deployment process is very straightforward. It's not complex. Only one associate deployed it, so not much effort was required.

There was no impact on the work that we used to do with performance testing when we moved to NeoLoad. Our normal performance testing was able to continue and it was going perfectly well. Return on investment was better because our deliverables were not getting impacted and it was cheaper, so return on investment increased a lot compared to LoadRunner.

NeoLoad is cheaper compared to other solutions. There are no additional licensing fees.

I would rate this solution 9 out of 10. I wouldn't give it a 10 because there are some issues that need to be resolved from the tool side.

Before using the tool, you need to understand the product you are going to design, the different types of performance applications that you are going to do performance testing with, and whether or not NeoLoad supports it. If yes, then go ahead. Otherwise, I would say go back to LoadRunner because whatever the application is, LoadRunner will support it.

We use Tricentis NeoLoad to test risk with API calls and make some small automation with the API calls using small Java scripts to take data from the file.

The solution is on-premises and we use a web dashboard which is SaaS but we use on-premise because we mostly work with the clients in its sensitive data that's required, like, only on-premise usage.

The solution is on-premises and we use a web dashboard which is Software as a Service. We prefer on-premise usage because we mostly work with clients dealing with sensitive data that requires on-premise usage.

The scalability, support protocols, and features of Tricentis NeoLoad made it easy and straightforward for us to complete the same tasks for our customers in the server cloud. We had to verify the certificate and attach the file within a protocol, which was a challenge in the server cloud.

From a functional perspective, the range of tools provided with Tricentis NeoLoad is perhaps the widest.

There is room for improvement with the support and community documentation as it can be difficult to find answers to questions quickly. 

I have been using the solution for six months.

The solution is stable. In the past six months, I have only encountered a few issues or bugs when the program crashed. I believe this may have been due to my own configuration settings.

The solution is scalable.

The technical support is not very good.

I previously used Cisco SMARTnet but the generators for reports were difficult to configure on the backend so we switched to Tricentis NeoLoad because it is more user-friendly.

The initial setup is straightforward. We deployed using the manual and did not integrate the solution with anything, such as the CID tool; it was only for small projects.

We used a 60-day trial with ten hours of work per month. The cost is increasing each year.

I give the solution an eight out of ten.

We have three people using the solution in our organization.

As long as the organization's requirements are met, I recommend Tricentis NeoLoad.

• Easy correlations
• User-friendly GUI
• All components in one place

It helped us to improve the performance of all our applications.

Almost everything about NewLoad looks good, but I would like to have an indication of the variables for error messages appearing in the error tab. This would help us identify the users triggering the errors.

I've been using it for a year.

We've had no deployment issues.

There have been no issues with the stability.

We have had no issues scaling it for our needs.

In my experience, customer service is good.

The times that I've contacted technical support have been good experiences.

I worked with HP LoadRunner in my previous projects. I started using NeoLoad when I joined a project that was already utilizing it.

I wasn't involved, but my colleagues have said that the initial setup is straightforward.

An in-house team implemented NeoLoad for the current project.

Compared to HP LoadRunner, NeoLoad is cheaper.

It's a good tool to use if you are not willing to spend lots of money on HP LoadRunner licenses.

It's got a number of valuable features, the best of which I think are the mobile testing capabilities. It allows for load testing to make sure users can experience apps properly and checks for performance issues on the server side.

I particularly like Neoload’s ease of use, built in monitoring and their extensive library of wireless devices and protocols. We are able to simulate various wireless communication standards such as 3G, 4G etc. with ease.

A lot of our customers are now using mobile technology, so it's provided us the platform and ability to do mobile testing, which in turn enhances their experience in our store.

NeoLoad’s lack of support for a complete protocol suite is somewhat limiting. While they do offer some protocols (Kronos, GWT V2.6) that others do not, they are not a one stop shop for protocol testing. Neotys is considered a forward looking company in the software performance testing world and legacy protocols such as RTE, SAP GUI and RDP are not included in their suite of protocols. I would like to see a more comprehensive offering of protocols with future releases.

I've used it for a little over a year.

No, not at all. This was probably the easiest part of it. iIt’s working out really well as it is not taking up much resource and it was very easy to install.

We didn't encounter any issues with stability.

No, but to be fair, we don’t have a very large license right now, so this hasn’t become an issue for us. If I plan accordingly, this shouldn’t be an issue over time either.

The questions that we’ve had for them, they have answered very quickly. The team here is very skilled and we've managed to resolve any issues we've encountered ourselves. But when we have reached out to them they have been very responsive and they've solved our issues quickly.

We haven’t completely switched over yet. We are also using HP Performance Center. We are starting to slowly migrate. The mobile capabilities and the cloud testing capabilities, HP are lacking. I think in the next three to five years we will totally migrate to Neoload.

We did it in-house.

We certainly have an ROI, but it is hard to put in to numbers. If I don’t do the testing, how much sales do I lose? Millions of dollars. I can definitely tell you it would be significant.

They have been very responsive. They have even made an on-site visit. It’s been very good, they have had recent turnover but they are managing the transition well and they have even had a top team member reach out to us.

In my company, we use the product on the cloud by just filling out and taking the subscription and then starting the application in our environment. In general, I am not aware of the technical areas associated with the product.

In Tricentis NeoLoad, there are some good options similar to what Apica offers. If you are planning extraction for eight or ten hours, NeoLoad can extract data from that particular time frame, which is something you can't do in Apica but is present as a good feature in Tricentis NeoLoad. With the tool, it is possible to compare NeoLoad test results against baseline and benchmark, and we can make the comparisons in the same window.

The debugging part of Tricentis NeoLoad takes time. Fixing the scripts takes time due to the problem with the debugging part. The debugging part of the product is an area that requires improvement.

I have been using Tricentis NeoLoad for a year.

My company uses the product on the cloud, so we have no issues with the tool.

At the moment, no one in my company uses the product. I used the tool when I worked with Vodafone's account, but at the moment, our company is not handling it anymore.

I have not contacted the support team of the product since I have not faced any issues with the solution.

The debugging part of Tricentis NeoLoad takes time. Fixing the scripts takes time due to the problem with the debugging part. The debugging part of the product is an area that requires improvement.

The solution is deployed on the cloud services managed by NeoLoad. Apache JMeter was also deployed on the cloud.

Not only for Tricentis NeoLoad but also any other tool would require some time to be installed.

Anyone can use the product, but it is important to create templates to focus on, especially if someone wants the business transactions to be easily captured in NeoLoad.

I rate the tool an eight out of ten.