Apache JMeter Reviews

I use JMeter for concurrency and some backend services. We are also using JMeter to apply our tests within our framework by sending some requests to JMeter to assess our application's performance. My company is developing software based on data analytics that produces insights and graphs like Tableau or Power BI. We use JMeter to test the graphs, how the tables are rendered, and how long it takes to render some applications. 

For instance, we have a problem with one of our clients that use some complex scenarios or queries that take too much time to retrieve from the application or render to the end user. We ran a couple of tests on the application using JMeter and spotted the service or request that was taking too much time.We alerted the developers about this, and they took the appropriate action to fix this problem. Afterward, we run another cycle of JMeter to ensure everything is working as expected.

There are two modules. The analytics module is the interface for the stakeholders and company decision-makers. The number of users isn't that big, unlike the retail applications websites. It's 500 users at most. The other module is deals with data volume. We are currently doing some POCs to check whether we will benefit from JMeter in this area because it's not a concurrency issue. It's a data volume issue.

We have a hybrid deployment because we are using JMeter to assess the performance of our products. If the product is deployed on the cloud, we use JMeter on the cloud. If it's on-prem, we are using it on-prem. At my previous company, we mainly used JMeter on-prem.

I expect that we will continue to rely more on JMeter, and we have multiple DevOps pipelines using JMeter to test another module in our application.

I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance.

The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter. It's working on this, actually.

It's an excellent plugin that you can use to record the scenario from Google Chrome, and it integrates easily into JMeter. They could also make it easier to generate the built-in report. Now, you run the tests and generate the charts in a separate column. The graphs and charts that display the test metrics could be better.

I worked with another tool called Web Performance Tester, and its interface is better than JMeter's. They have intuitive graphs while you are running the tests, so you can see how things are going. It shows you the number of concurrent users logged into the system, the number of failures, response times, etc.

I worked in performance testing from 2011 until 2019. I was working with another tool, but in the last few years, I started using JMeter for a couple of projects.

It's hard to say definitively. When there's something wrong with JMeter, we're unsure whether it's something in the device because there are lots of requests coming from the machine. Maybe we didn't use it that many times from our local server. There are some bottlenecks, but in many cases, we only need to restart JMeter, and it works fine.

JMeter is scalable. I did a test using it in a distributed environment with more than one client. We used JMeter to load from the server and also the plugin tool. It works fairly well in a complex environment. I still need to research the maximum concurrent JMeter can handle. Is it 1 billion or 10 million? 

In my work experience, I need to load maybe 1 million users. For example, most of my work is on the backend of e-commerce websites with a maximum of 1,000 users. We have many clients, but the load is not that big. 

I don't think JMeter has technical support because it's an open-source application, but there is support for third-party apps that use JMeter. For instance, BlazeMeter has good technical support.

I have more experience in a tool called Web Performance Tester, but it has some limitations. For example, it doesn't support some forms. A long time ago, their support was better, but lately, their support hasn't been so good. They always ask us to switch to the consultancy service and they don't give us the solution. If we face problems, they always tell us, "Okay. You can consult us." This is not a good practice, actually.

Also, Web Performance Tester isn't well-known, so some of our clients aren't confident in it because it doesn't have a reputation.  They were more familiar with JMeter, and it's something I've used.  However, I have no problem with Web Performance Tester. It's hard to compare the two tools because it's not apples to apples. 

I joined this company in October, and they had just started using JMeter. Previously, they were using a tool that they had developed in-house. They found that JMeter has more capabilities and specificity than the tools they were using.

Setting up JMeter is straightforward. It doesn't need an installation like other tools. There is a batch file for Windows. Around four people in my organization are responsible for maintaining and managing JMeter.

JMeter is a free open-source tool. There are some third-party tools built on top of JMeter that have a license or something like BlazeMeter. I think you can also purchase some additional services.

I rate Apache JMeter eight out of 10.  Before using JMeter, you should find some tutorials because you need to be trained to use it. It's not a plug-and-play tool. This is what I did. Spend some time researching JMeter's capabilities. We met a lot of people who knew about JMeter, but when you tell them that it can do something, they're not aware of it. 

For instance, through taking some courses, we learned that JMeter has distributed performance capabilities. Some people still know that you can't make these recordable scenarios. This information hasn't reached everyone in the IT market.

I mainly use JMeter to capture the traffic of the most-visited page to see how much load a particular page is getting and how many users are using that page for a particular amount of time. I've also used it to capture APIs for particular pages.

The most valuable features are the ability to capture the entire traffic of particular pages and the proper readability of entire pages and entire APIs.

One of the drawbacks of JMeter is that it can't handle a large amount of load, which forces us to switch to other tools when we need to load more than a 5,000 or 10,000 user load. In the next release, I would like JMeter to be more compatible with other languages in the market.

I've been using JMeter for six to eight months.

JMeter can't be used in the long run, so I'd rate its stability as five out of ten.

I would rate JMeter's scalability as seven out of ten.

Apache's technical support is pretty good, I've had no issues with them.

The initial setup was straightforward and took about five to six months.

I used an in-house team.

JMeter is open source, so there are no licensing costs associated with it.

I evaluated SoapUI, Postman, and Visual Studio. JMeter was more reliable compared to these options.

I would definitely recommend JMeter in terms of usability. If you're using AngularJS as a language for testing UIs, JMeter might not be a good idea. I'd rate this solution as six out of ten.

On-premises

We use Apache JMeter for performance testing, including the response time of our websites for multiple users.

The product is specialized in performance testing. It gives accurate results and recommendations that we can implement to enhance the performance of websites.

Apache JMeter could be a more user-friendly product from the end user's perspective. It requires someone with technical knowledge to administer it. This particular area needs improvement.

We have been using Apache JMeter for a year.

I rate Apache JMeter's stability a ten out of ten.

We have around 12 Apache JMeter users in our organization. I rate the product's scalability an eight out of ten.

The technical support team responds quickly to queries.

The initial setup process is very complicated for the on-premises version. It would be better to use the cloud version.

The product has reasonable pricing.

It is a complete tool that integrates lots of functionality. I recommend it to others and rate it a nine out of ten.

On-premises

We primarily use the solution for performance and delivery testing.

It's a free tool. It's a very common tool. There's a lot of support in the community for this. It mainly supports web API testing.

The UI could be better. It can have some Reach UI also, which would be helpful, and maybe a relatively simpler way of using it. It needs simple modules. There are quite a lot of things which are kind of abandoned, so they can definitely improve on it.

Integration with some of the other features should be managed. However, it's open source, so there is not much to complain about there.

It's an open-source tool; we cannot ask for additional features really.

The product could use some kind of filtering and monitoring and different degree of dashboards and analysis. If that can be provided, that would be very, very helpful.

I’ve used the solution since 2009.

The solution is relatively stable. There are no bugs or glitches, and it doesn’t crash o freeze. It’s reliable.

It can scale. However, what others say about scalability is that you need to have some proper calculations to be done first.

We have dealt with support. Technical support is found in more technical forums. It’s open-source, and communes are where you go for information. We have received good responses when we have inquiries. There are quite a lot of forums in general available.

Now, as we have grown as an organization or as a team, there are still questions such as, "What are the limitations of this tool?" And we put that to JMeter so we can learn what is best for the maintenance.

The installation can be a bit complex. There are quite a lot of things and issues if you go deep into it and if you're setting up JMeter. For example, direct script captioning is slightly different. It's a bit more complex, the correlation parametrization. Setting up the workload model can be complex as it is based on a Java service mechanism.

It becomes a bit challenging to manage. If I want to put a 50,000 user load, I have to be very careful how the memory is utilized. I must be very aware of the underlying system's capability to execute this. I have to make it into multiple nodes and run them in parallel. There are some calculations, and there are some good power processes that will be required when you are using JMeter. You need to understand its limitations and load work them on.

The solution doesn’t really need maintenance. They're open-source tools, so we don't expect any maintenance. What we typically do is we pick up a particular version and understand the limitations of it. We then play within those limitations.

We handled the initial setup in-house. I handled the setup myself.

The solution is free to use.

We may be using the latest version. I don’t know the exact version number.

It is on-prem. However, we can put it on the cloud as well. We install it on any machine, so it can be a local or cloud-based machine.

I’d recommend JMeter. One critical piece of advice is to plan properly. For JMeter, planning and having an understanding of the limitations are important. If you play within those limitations, it is a really great tool. There are trade rules that apply to a lot of things that we use. However, there's plenty of material available so far. Whoever is doing it, can plan it accordingly. They can create those tests, and execute those tests. That said, at the same time, be aware of the scalability of the JMeter. For example, a single JMeter line, which is running, can take 1,000 to 2,000 threats, not more than that. Again, it depends on the available hardware. If you want to scale to, say, one million or something, different parts of planning are needed.

I’d rate the solution eight out of ten.

On-premises

We use JMeter for performance testing. Our testing team has 50 to 60 people.

JMeter lets us generate virtual users and T-load, per our requirements. It's easy to configure and adjusting the virtual users according to the DPS we want to achieve.

It should be easier to combine multiple scripts. If you have multiple scripts, you need to write a new script to combine those scripts. The virtual user generator is slow.

We've been using JMeter for six months.

With an on-premises deployment, we run into some performance problems when we have a high load. It's not that stable if we increase the scripts or the load. But that could be something in our environment.

JMeter is an on-premise solution, so it has limited scalability. We are looking for a cloud-based solution that is scalable and elastic. 

I rate JMeter six out of 10.

On-premises

We use Apache JMeter for the load generation scripts for performance testing.

The thread groups, samplers, and listeners, which are all determined by the script's requirements, are the most valuable features of this solution.

We use many plugins to customize our scripts, which is its main purpose. We wanted to be able to use a larger variety of customizable plugins to meet our needs. Along with our, JMeter, you would use a variety of plugins.

The number of customization plugins should be increased.

There could be improvements in terms of memory utilization. We are going to migrate away from JMeter in the near future.

The data storage should be improved.

Scalability could be improved.

It should support more protocols.

I have been working with Apache JMeter for three or four years.

We use version 5.0

We are using the cluster mode because one node is definitely not enough. Scalability can be improved.

When compared with other solutions, it's not as good, which is the reason we are moving to another solution.

Scalability must improve.

Generally, we use open forums, to resolve any issues we may have.

I also work with BlazeMeter.

I worked with Apache JMeter, from the beginning.

The initial setup is straightforward.

We have a small team to maintain this solution.

We completed the installation on our own. It was completed in-house.

Apache JMeter is an open-source solution.

We don't use the paid version of this solution.

Everything is included, and there are no additional costs.

I would suggest that instead of using a GUI-based implementation, try to make it code-based. Try to replicate the configuration. The plug should be job-ready, and ready to be integrated as well. Rather than having a UI, and limitations.

Three or four years ago, I would have given it a seven or eight, but now that there are more powerful competitors, I would give Apache JMeter a five out of ten.

My main use case of JMeter is for web application performance testing as well as for API performance testing. We are customers of Apache. 

The benefit of JMeter is that it does our performance testing and provides a report without the need to spend money on a licensed tool. It's a significant benefit for us and for the project we're currently working on.

The fact that the solution is open source makes a big difference as we're working on a low-budget project. It's quite user-friendly and easy to use.

There are issues with the plug-ins which you need for reporting purposes as they make the reports quite heavy so you have to run them in non-GUI mode. If you go above the 200 user mark, the application creates a bottleneck and that's one of its major drawbacks. It means you have to run with a master-slave configuration with one system being the master, and multiple slave systems. It's not ideal and I think it could be simplified with a UI that provides direct configuration. In addition, the solution doesn't support SIP applications and some other protocols.

JMeter is not designed for high loads, if you overdo it the tool becomes a bottleneck. Unless you're using JMeter in multiple systems and all the systems are connected through a particular LAN, there is a limit.

Because it's freeware there's no official tech support but you can raise a request on the JMeter site. We haven't had many problems. 

We were previously using LoadRunner for this project but moved to JMeter because it's a freeware testware.

The initial setup is quite easy. There is no deployment process, you download a file from the web application service from your JMeter site, and you can use it. Any user with a little bit of knowledge can do it. The download takes about 15 minutes so you can be using JMeter in about half an hour. From time to time there are version upgrades but they don't affect existing assets. We have four people using JMeter and they don't have any issues. 

BlazeMeter works on top of JMeter and there is a small cost factor to purchase that. It basically gives a slightly more advanced JMeter. 

If you're working on a low-budget project and don't have dependencies of a huge number of users then this is the perfect tool. If you have 2,000 or 3,000 users then it's probably best to look at other options.

I rate this solution eight out of 10. 

On-premises

We use Apache  JMeter for API testing. Our latest task combines test cases with our CI/CD pipeline for deployment to Kubernetes. Tests are developed locally and automated for the build using standard Unit and Integration testing techniques. Once deployment is successful to Kubernetes, we develop the JMeter tests for automation in promotion scenarios. Thereafter, once a deployment is successful, we can automate promotion to the next level based on test success.

Test automation moves our organization close to rapidly deploying products. Unit and Integration testing is easy to automate, and most organizations perform these as part of their day-to-day operations. However, end-to-end testing, smoke testing, load testing, and performance testing are much harder to automate. Apache JMeter has aided in that challenge.

While there is a User Interface, the scripting ability is highly beneficial and is easy to use. Tests can be added to a CI/CD Pipeline for integration with testing and deployment scenarios once finalized and operational.

This is a difficult question to answer. On one side, JMeter is very flexible and allows for a high amount of customization. On the other, some tasks are common enough that it merits simplifying the process.

Authentication for API testing could use improvement. Currently, it is a multi-step process to call, extract, and utilize a bearer token securely for API calls. This process is becoming a common enough task that a "wizard" for creating and consuming popular authentication models is merited.

I have been using this solution for about six years.

Apache JMeter is stable, and I personally have not encountered any issues. Depending on the size of test runs, one might need to adjust their JAVA settings to align with the test requirements.

Its scalability works. It is a typically Java run. Therefore, it is limited only by what you can do in Java in terms of scalability.

Developers write tests, verify tests, and maintain tests using version control. They identify and tag each to ensure they are appropriately labeled for test purposes (E.g., unit testing, integration testing, performance testing, and the like). Unit and Integration test coverage is normally high. However, we require testing from outside of the system, and JMeter allows us to create tests automating this process.

Apache JMeter utilizes community support. It is well-documented and has an active community. As far as I know, there is not a "pay-for-support" option.

I have used Postman in collaboration with other developers. However, I prefer Jmeter only out of personal familiarity and not for any technical deficiencies of Postman.

Apache JMeter setup is easy. However, there is a medium-to-heavy learning curve for developing tests and getting started using it for practical uses. Depending on its intended uses, there could be a significant configuration task for a given set of tests.

Apache JMeter is under Apache License, Version 2.0 licensing. Understanding licensing requirements is important for the implementation of any tool.

Understand the use case. Choosing the correct tool for any task is always a challenge. Jmeter offers a significant amount of flexibility and will work for a lot of solutions. Jmeter requires a commitment to learning for optimal operation; without that investment, tests may not yield the appropriate outcomes.

Public Cloud

Amazon Web Services (AWS)