Apache JMeter Reviews

Our company uses the solution for performance testing on web applications, API load testing, and automation of APIs and web applications. The solution provides three streams of use. 

We also integrate Selenium with the solution to perform automation testing. 

The solution has good transition controllers and distributed testing. 

The solution's setup could be easier and security could be improved to minimize vulnerabilities. 

I have been using the solution for four years.

The solution is used to test the stability of other applications through a three-step process. 

Load tests look at the CPU, memory, and APM tool and assess the current load to determine if it can be increased from 60% to 70% for example. You continuously monitor the APM to see fluctuations in memory and CPU usage. If load tests are successful, then you move on to stress tests. 

Stress tests find bottlenecks and determine how many users the application can handle within the current infrastructure. Baseline tests can be performed to determine results such as response time when increasing users. 

Endurance tests determine if an application is stable and for how long stability can be expected. For an endurance test, you set up a particular user load and run it for at least eight hours but up to twenty-four hours. During this time, you see how the application is behaving and whether the CPU and memory are stable. This will allow you to assess overall stability without OOM leakages. 

The solution determines if there are lags in application performance. For one project with 500 continuous users per hour, we used the solution to find that there were only two lags from 50,000 samples generated.  

The solution is used to perform baseline tests for applications and compare them to SLAs to determine user load and scalability. 

I have not needed technical support. 

The initial setup can be complex depending on the application you want to test. 

If the application is straightforward without much logic, then setup is easy and only requires a simple transaction controller. This scenario is common for users who log in, perform simple actions within 15 minutes, and log out. 

Applications with internal logic require scripting that includes sample JSON extractors, pre- and post-processes, transition and throughput controllers, and precision throughput timers. This scenario is common for users who perform multiple actions over a longer period of time, is not straightforward, and involves multiple transaction controllers. 

Our company implements the solution for customers.

First, we ask our customers about critical work flows and categorize them based on user loads to create a workload model. Then, we create scripts and proceed with testing. 

Often, we request access to a remote desktop for installing scripts and resetting networks. We follow the workload model to initiate load, stress, and endurance tests. 

The solution is open source with no user fees or licenses. 

The solution is comparable to LoadRunner because it works with multiple plugins but is free to use. 

I recommend the solution and rate it a eight out of ten. 

The solution is open source so on occasion there are security vulnerabilities. 

We are in the financial industry in India and carry out performance testing. We deal with  SWIFT messages and financial messaging systems. A few years ago we had 2 million transactions in five hours; today it's 2 million in two hours. To measure these performances, I use JMeter for regression. We have Exadata, Exalogic infrastructure. We are users of JMeter and I'm a technical specialist. 

This tool is very user-friendly and easy to use. It's open-source so there are no costs involved for the non-production environments. It's easily available to anyone who wants to use it and it has all the features required for performance metrics.

I think it has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems when we are working in certain specific environments that have a proxy setup. When we want it to do a record with some new scripts, there are some challenges there.

I've been using this solution for two years. 

The solution is stable and easily scalable. 

The support is very good, with good technical teams. Whenever we get stuck they support us and provide solutions. 

The initial setup took only 10 minutes but there were some complications with the product initially having less features. That has been greatly improved. It's easy to do maintenance of the test tools which we develop, and it's easy to maintain as a product. We have around 20 users of this solution. 

It's open-source, so there's no cost. If you need some support from the vendors there will be a charge for that. 

I would recommend this solution. Most of the support, guides, and tutorials are available on YouTube. 

I rate this solution an eight out of 10. 

The most valuable feature of this solution is being able to launch many requests and scheduling simulating human interactions with the application. There are some other plugins for monitoring and collecting data for JMeter but we don't use them. 

If JMeter could integrate with the EPM solution, it would be great. It could also be improved by offering more integrations for security. For example, most applications are secure with OpenID Connect protocols.

I have been using this solution for more than 10 years.

This is a stable solution. 

We previously used Katalon. 

The initial setup is straightforward. 

I would rate this solution an eight out of ten. 

We are in the service industry. We implement it for our customers. We recommend the right tool and set it up for them. So, I've not had any hands-on experience in my current role, but I have a good understanding or a fair idea of the tool's capabilities. I have a team that takes care of the technical aspects.

It is an open-source solution. So, typically when you don't want to make a heavy investment, and you want to do some level of performance testing, Apache JMeter is used. 

It is typically on-premises, and it has also been on the public cloud. It could be Azure, or it could be AWS. It is very rarely on GCP.

It helps them to look into several parameters. For example, when you have certified test cases that are predominantly repeated on an application, you can execute the same thing with increased load. You can see how the application responds and if there is an impact on the response time of the application. You can confine it to certain parameter conditions and then start making changes to see how it performs. You can see where the RAM or CPUs are stagnant and not increased.

The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features.

It is easy to use. If you want to test your application out and not incur a lot of costs, it is probably the best tool.

Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports.

I have been using it for about four to five years.

JMeter is probably good for lower loads. It is not comparable to LoadRunner when it comes to higher loads.

It probably scales up to a few thousand users but not beyond that.

Because it is an open-source community, their support is probably average. It won't be like the support for a commercial product. I would rate it a six out of ten.

LoadRunner is one of the prominent tools. It was formerly HP, and now it is Micro Focus. It has good capabilities and features. It also has decent reporting capabilities. Because of the brand and the capability, it was probably chosen by most of the Fortune 500 clients that we work with. There are also some startup communities or organizations that ventured into other solutions, such as JMeter.

In terms of comparison, primarily, there are three to four parameters. The first one is the ease of use. The second one is about the protocols that need to be tested, whether it's web or API, HTTP, HTTPS, and all the native things. The third one is in terms of flexibility in setting it up and executing, and the fourth one is in terms of monitoring the execution and reporting pieces. Those are the key parameters for pros and cons. LoadRunner gives you a lot more capability and flexibility, but at the same time, it also consumes a lot of resources. JMeter is relatively simpler, cheaper, and easier to use.

I don't have hands-on experience with it, but based on what I have heard from people, it's pretty straightforward in terms of the setup. 

The setup probably takes a week or two, and then the execution is probably a three to four weeks exercise.

We are in the services business. Clients give us access, and we set it up there.

It is certainly good for testing out the applications for performance testing, especially when you have to test them out frequently and make sure that they are good for at least a few thousand users. It has a decent ROI.

It is open source. There are no licensing costs associated. If you need enterprise support, you'll probably end up paying for a license.

You would also factor in the infrastructure cost, but that's not significant.

I would rate it a seven out of ten. It is a decent choice from a small-scale perspective, but reporting could be better. If you want to get some performance testing done without spending money, JMeter is probably the best tool. It doesn't have the best reporting, but it is quite a handy tool.

I am using this solution for a banking use case. We have an older version of this solution. We are planning to upgrade to a newer version.

A lot of things are valuable. It is free. It has a lot of features, such as report generation and integration with CI/CD, which makes it very competitive with the other paid solutions available in the market. It is a good solution.

They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. 

In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. 

They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available.

I have been using this solution for almost three months.

It is stable. There is good support, and fixes are also available immediately.

We have not scaled it, but it is scalable. We plan to increase its usage. 

I didn't have to call Apache because if anything comes up, we are able to find a solution in the community.

Its initial setup was straightforward.

We are using the free version, and if required, we can easily switch to the other version.

I would definitely recommend this solution if it suits a use case. It is good, and it also looks promising. We plan to keep using it in the future.

I would rate Apache JMeter an eight out of ten.

This is a cost-effective tool, which provides people the ability to do POCs without worrying about costs.

Apache JMeter provided us with the opportunity to test applications for clients who did not have the budget to employ heavy-duty tools like LoadRunner.

Additionally, due to ease of installation, it can be installed in any client network, without the requirement for port opening.

API testing, Database Testing, and MQ testing can be done with ease.

The tool should be made a bit more robust, and better support should be made available.

I set a script reporting and execute test report and analysis on almost all stages of the performance cycle except in regards to requirements and the environment.

we saved a lot on licenses for commercial tool

The features that helpful are the coding and the opportunity to work with the SQL directly in order to report on the data. You can collect all the test data from DB with these and then do post test execution metrics collection as well(if you need DB)

RegEx Extractor needs improvement. It's a headache for many people. The solution could use some sort of educational features to offer tips and hints to help users navigate it better. They should improve the manuals and help files. I've searched the internet for answers over the past year or so, and I haven't come across anything that is helpful. The community help files are pretty good, but their own help files are not.

In the next release, it would be helpful to offer more flexible graph handling to be able to combine different metrics into one graph.

quitre stable but keep an eye on RAM consumption and smart execution tips (can see on their site)

Scalability is good, though you need to be aware of distributed and non guis test execution modes. 

It's a free tool, so they don't offer technical support. If you need help, you need to Google and search for answers on your own.

I've used LoadRunner in the past, among other solutions. I switched to Apache because it is free. Other tools are much too expensive and can cost up to $50,000 a year if you are looking at commercial options.

Now Neoload probably the best one (better than Load runner as per my subjective perception) but it's a way too costly 

please take a look at ROI and scal of performance tests. For big projects and organizations consider Load Runner or Neoload, but if you have small project, most likely you wil lstop on Apache Jmeter. Also it depends on the protocol support. But because of big community, Apache jmeter has constant income of new plugins developed

For people thinking about implementing the solution, I would advise that they know what protocols they want to work with. I would recommend doing an evaluation of script maintenance. You need to see how stable your automation disk and performance is. 

I would rate this solution nine out of ten. The solution offers a lot of plug-ins and a huge, continuously developing community that is regularly offering new features and plug-ins.

I use the solution in our organization for load testing.

The solution's automation capabilities are one of the best parts of the solution. Due to process automation, I don't have to prepare reports, making it the perfect solution. Whenever I require a report, the report gets generated by the solution, and then afterwards, I can look at the reports before copying and pasting them.

At times when we overload the application, it gets stuck. Especially with load testing, it gets stuck when we deal with huge numbers of 100 to 1,000-10,000 or 2,000 to 20,000. I believe that we should be able to hit around a million people without any issues. After the solution gets stuck due to overloading, we have to restart our computers. In short, the solution keeps crashing.

Apache JMeter is a simple solution, and I have been using it for seven years. Also, I am using the solution's latest version.

After an update in the application, the stability gets hampered if we continue to use its non-updated version, causing the solution to crash. So, once I faced that problem since I was not updating the application, it crashed. Back then, I wasn't aware that an update was available for the solution, which I had to apply to the solution. So, for two or three days, I tried to solve the issue with the solution. Later, I found that I just needed to update the application. So, after I updated the application, it started working fine. I rate the solution's stability a seven out of ten.

Since Apache JMeter is a scalable product, I rate the solution a ten out of ten. I am the only person using the solution in my organization.

Since I have experience with the solution's customer support team, I rate the technical support of the solution an eight out of ten.

Positive

On a scale of one to ten, where one is difficult, and ten is easy, I rate the solution's initial setup a ten out of ten.

The solution's deployment process took only a couple of hours. I believe it took me around an hour or so at the beginning during the initial setup phase. Right now, I can deploy the solution in fifteen to twenty minutes.

Maintenance for the application is only required when the application crashes due to its overuse.

Since we are using an outsourced solution, it is not paid for by our company.

My words to those planning to use the solution is that if the crashing issues in the solution get resolved, it is one of the perfect solutions in the market. Also, I have not used the paid version, so I have no idea how it works. It's a very resourceful solution.

I need to use the paid version to be able to comment on it. Since we are using the free version in a company, even if the solution makes any mistakes, we ignore those and go ahead.

Overall, I rate the solution a ten out of ten.