Apache JMeter Reviews

My main use case of JMeter is for web application performance testing as well as for API performance testing. We are customers of Apache. 

The benefit of JMeter is that it does our performance testing and provides a report without the need to spend money on a licensed tool. It's a significant benefit for us and for the project we're currently working on.

The fact that the solution is open source makes a big difference as we're working on a low-budget project. It's quite user-friendly and easy to use.

There are issues with the plug-ins which you need for reporting purposes as they make the reports quite heavy so you have to run them in non-GUI mode. If you go above the 200 user mark, the application creates a bottleneck and that's one of its major drawbacks. It means you have to run with a master-slave configuration with one system being the master, and multiple slave systems. It's not ideal and I think it could be simplified with a UI that provides direct configuration. In addition, the solution doesn't support SIP applications and some other protocols.

JMeter is not designed for high loads, if you overdo it the tool becomes a bottleneck. Unless you're using JMeter in multiple systems and all the systems are connected through a particular LAN, there is a limit.

Because it's freeware there's no official tech support but you can raise a request on the JMeter site. We haven't had many problems. 

We were previously using LoadRunner for this project but moved to JMeter because it's a freeware testware.

The initial setup is quite easy. There is no deployment process, you download a file from the web application service from your JMeter site, and you can use it. Any user with a little bit of knowledge can do it. The download takes about 15 minutes so you can be using JMeter in about half an hour. From time to time there are version upgrades but they don't affect existing assets. We have four people using JMeter and they don't have any issues. 

BlazeMeter works on top of JMeter and there is a small cost factor to purchase that. It basically gives a slightly more advanced JMeter. 

If you're working on a low-budget project and don't have dependencies of a huge number of users then this is the perfect tool. If you have 2,000 or 3,000 users then it's probably best to look at other options.

I rate this solution eight out of 10. 

On-premises

We use Apache  JMeter for API testing. Our latest task combines test cases with our CI/CD pipeline for deployment to Kubernetes. Tests are developed locally and automated for the build using standard Unit and Integration testing techniques. Once deployment is successful to Kubernetes, we develop the JMeter tests for automation in promotion scenarios. Thereafter, once a deployment is successful, we can automate promotion to the next level based on test success.

Test automation moves our organization close to rapidly deploying products. Unit and Integration testing is easy to automate, and most organizations perform these as part of their day-to-day operations. However, end-to-end testing, smoke testing, load testing, and performance testing are much harder to automate. Apache JMeter has aided in that challenge.

While there is a User Interface, the scripting ability is highly beneficial and is easy to use. Tests can be added to a CI/CD Pipeline for integration with testing and deployment scenarios once finalized and operational.

This is a difficult question to answer. On one side, JMeter is very flexible and allows for a high amount of customization. On the other, some tasks are common enough that it merits simplifying the process.

Authentication for API testing could use improvement. Currently, it is a multi-step process to call, extract, and utilize a bearer token securely for API calls. This process is becoming a common enough task that a "wizard" for creating and consuming popular authentication models is merited.

I have been using this solution for about six years.

Apache JMeter is stable, and I personally have not encountered any issues. Depending on the size of test runs, one might need to adjust their JAVA settings to align with the test requirements.

Its scalability works. It is a typically Java run. Therefore, it is limited only by what you can do in Java in terms of scalability.

Developers write tests, verify tests, and maintain tests using version control. They identify and tag each to ensure they are appropriately labeled for test purposes (E.g., unit testing, integration testing, performance testing, and the like). Unit and Integration test coverage is normally high. However, we require testing from outside of the system, and JMeter allows us to create tests automating this process.

Apache JMeter utilizes community support. It is well-documented and has an active community. As far as I know, there is not a "pay-for-support" option.

I have used Postman in collaboration with other developers. However, I prefer Jmeter only out of personal familiarity and not for any technical deficiencies of Postman.

Apache JMeter setup is easy. However, there is a medium-to-heavy learning curve for developing tests and getting started using it for practical uses. Depending on its intended uses, there could be a significant configuration task for a given set of tests.

Apache JMeter is under Apache License, Version 2.0 licensing. Understanding licensing requirements is important for the implementation of any tool.

Understand the use case. Choosing the correct tool for any task is always a challenge. Jmeter offers a significant amount of flexibility and will work for a lot of solutions. Jmeter requires a commitment to learning for optimal operation; without that investment, tests may not yield the appropriate outcomes.

Public Cloud

Amazon Web Services (AWS)

When I was last using JMeter, we were simulating 200 concurrent users and evaluating performance based on transaction times. We were defining SLAs based on the results.

Essentially, we created load scenarios and testing different ones using different workload models.

The recording and playback functionality is helpful.

The reporting is not very good.

When we run with multiple users, it takes a lot of memory.

With respect to the recording and playback functionality, the auto-correlation parameterization is not easy and should be improved.

I have been using Apache JMeter for about four years.

There are issues with stability when running with multiple users because it consumes a lot of memory.

Scalability is fine, although it is important to remember that JMeter doesn't run on its own. It needs to work with load-generations such as BlazeMeter. LoadRunner is the same in that you need a cloud-based infrastructure to run it.

There is no official support. There is a forum where you can ask questions and they respond to you, but the technical support that we have with LoadRunner or NeoLoad is not available.

I have used many similar solutions in the past such as New Relic, AppDynamics, NeoLoad, and Micro Focus LoadRunner Enterprise.

JMeter is not as good as LoadRunner or NeoLoad, and it isn't as easy to use, but it's okay because there is no cost. LoadRunner is too expensive, in my opinion. NeoLoad is cheaper, although not significantly.

From what I have seen, many companies are adopting JMeter because it's free. Especially in Canada, using JMeter seems to be the new trend. Some companies are choosing NeoLoad over LoadRunner because it is easier, faster, and cheaper. Whatever they need to do can be completed quicker. The main problem with NeoLoad is that obtaining resources is harder.

Given all of the choices, my preference would be to implement NeoLoad.

The initial setup is straightforward. I would not say that it is complex and if you already have the file downloaded then it will only take about half an hour to deploy.

I took care of the deployment myself.

I was using the free version of the software.

My advice for anybody who is considering JMeter is to just install it and try it. Creating scripts is a different process when you compare it to LoadRunner or Neoload. There is different terminology compare to these two products, so if somebody has not used JMeter then it may seem difficult at first.

I would rate this solution a six out of ten.

On-premises

Regardless of, whether it is used for personal performance, load, volume, UI test, or functional test Apache JMeter is the best tool on the market.

It is cost-effective and simple to use.

Take, for example, a performance test with, the Micro Focus performance test tool, and compare it to Apache JMeter. Apache JMeter is far less expensive than HP Performance Center.

When you run tests with JMeter, it generates test version five, which is extremely large. Also, when you have a large number of tests to run, it requires a large size or memory size, which basically means it consumes a lot of memory. It would be helpful to come up with a way to be able to use Apache JMeter in a way where it did not use as much memory. It will be much easier, and beneficial for the individual to run it on their own machines rather than having a high-end infrastructure, more CPUs, or more memory that has been consumed by Apache JMeter.

I have been working with Apache JMeter for two years now.

I am working with the latest version.

It's on digital platforms rather than the cloud.

I am absolutely satisfied with the stability of Apache JMeter.

Apache JMeter is a scalable product. We are pleased with the scalability of this solution.

When we have encountered difficulties and reached out to technical support, they respond very quickly and most of the things are readily available with them, for all of the challenges or support that are required. If I had to rate them from one to ten, I would rate the technical support an eight out of ten.

Positive

It is very easy to set up. There have been no issues, it's hassle-free, and it doesn't require a lot of expertise or specialization to configure and set up JMeter. 

It is very straightforward and very easy. That is something that anyone can do.

Our team completed the deployment. It only requires one person and takes no more than two to four hours.

In terms of maintenance, we only require one person.

When you compare it to HP because prior to using this Apache JMeter, we used this HP Performance Center, and in terms of ROI, while I don't have exact figures, when it comes to HP Performance Center, we have nearly five times more. You can say that it's five times less of an investment and five times written on the investment by using Apache JMeter. With Apache JMeter, you receive five times more. It's higher.

It's a yearly license.

I would rate Apache JMeter an eight out of ten.

On-premises

We're primarily using the solution for its performance. It's on our data center.

The tool has been a great help to our organization, which is focused more on quality testing as opposed to performance testing. It's provided a lot of insights. With the digital transformation that is currently happening, the need for that shift was rather critical.

The performance of the solution is excellent.

They have designed the product so that it is very easy to configure. 

You can basically do anything you like with the product. It's not very restrictive.

We like the fact that the technology is open-source.

The user interface could be improved. If they had better UI, it might make it easier to use.

You really need a technical team in order to really utilize the product.

The scalability could be better, or the process of scaling itself could be a bit more clear.

We've used the solution for a couple of years at this point.

The stability is quite good. There haven't been any issues with bugs or glitches. It doesn't crash of freeze. It's very reliable.

Nobody ever has used any tool to its full capacity, including this one. Half of the time it's how you set up and how your environment is. We can easily scale on cloud. So far, we're worried about its ability to scale. Our setup is basically a hybrid cloud where we can have private data centers and we can know exactly where each is located. However, the scaling is a big concern for me. I'm looking at BlazeMeter as well, and, although I've only really read about it, it looks like it scales quite well. With scaling, I'm sure I'll find more complexities, especially if I look into kiosk testing. I'll want to explore further test cases.

Currently, those that are on the product are mostly testing communities approaching it from a performance aspect. Every project is different and therefore the number of users goes up and down.

There isn't really a need for tech support with JMeter. There is a vibrant community, which is a good way for a lot of users or developers to go on and post a question and get an answer. The team prefers it this way. It's a positive aspect of JMeter.

I have some experience LoadRunner, however, it is expensive for what it offers. This product is open-source which makes it affordable.

The initial setup is pretty straightforward. My team was very hands-on. They didn't struggle with the UI at all and it wasn't too complex for them. That said, if a person with lesser skill levels, maybe someone who has a lower skillset, may struggle a bit with the implementation. It's geared slightly towards more technical users.

The product is an open-source solution.

I understand the Apache also has something called BlazeMeter, which is something I recently looked into.

I'm not quite sure which version of the solution our team is using right now. It's likely the latest version. We did the upgrade on our tools in 2019. It's open-source, and therefore everybody picks up the latest one.

As an open-source tool, there's a good community surrounding it. If you have automation frameworks or a DevOps pipeline, you can connect that and easily configure everything to streamline processes. Due to the fact that it's open-source, you don't have to wait to get the approvals and the budgets in order. That part alone can take months to sometimes years. If your company has the time, they should review the tools. If it seems to fit your organization, I'd recommend trying it out. 

Overall, I would rate the solution seven out of ten. It's a good open-source tool that you can configure easily, and it's very competitive in the market in terms of usability

Hybrid Cloud

We engage in performance testing using JMeter based on the client's requirements. We decide the endpoints for JMeter based on the domain and application requirements. The requirements, like 50 users per minute or 500 users per hour, guide our performance testing framework.

JMeter has allowed us to effectively verify that, based on pre-requirements, performance standards are met. Since it's an open-source tool, it also allows us to reduce the expenditure on performance testing tools.

The different listeners in JMeter provide various reports and results. It's very useful for performance and load testing. Its compatibility with external tools, such as the ability to import scripts from Selenium and Azure DevOps, makes it a robust and flexible option.

I would recommend the inclusion of plugins for AI in JMeter. Some features still require a certain knowledge level to use effectively, and making these features more user-friendly would be beneficial.

I have not been a direct team member but have managed a team working on JMeter for almost eleven years. In my nineteen years of experience, I have engaged with performance testing, with JMeter being part of our toolkit.

Stability is not a concern for expert users of JMeter. However, stability can be an issue for beginners.

JMeter is widely used and very scalable, rated between eight and nine in terms of scalability. It can also integrate with cloud solutions like Microsoft Azure.

There are many online blogs, forums, and video tutorials available for JMeter, providing substantial support for both beginner and intermediate levels.

Positive

LambdaTest and BrowserStack were part of our working user experience, but not extensively more than that. I initially worked with other performance testing tools like LoadRunner.

The setup for JMeter is very easy. There is no installation required; it is simply a package that needs to be imported and can be set up within half an hour or less, depending on the download speed.

Our clients cover the necessary licensing costs, particularly when using cloud services like Microsoft Azure.

JMeter itself is an open-source tool, so there are no licensing costs associated when running on-premises. Client-covered SaaS licensing, such as with Microsoft Azure, does not involve any costs on our side.

It's important to be very skilled in manual testing concepts before starting with performance testing. Due to JMeter's advanced level of techniques and skill requirements, extensive hands-on experience with real-time projects is needed. It generally takes three to four years of practical experience to master.

I'd rate the solution eight out of ten.

Hybrid Cloud

Microsoft Azure

We use Apache JMeter for load testing, where we provide the throughput time. Apache JMeter allows me to give any number of users to put the load in a particular time frame. With Apache JMeter, we won't need multiple tools with the application or to give a load for the application.

Apache JMeter's UI can be made more colorful.

I have been using Apache JMeter for two years.

I rate Apache JMeter ten out of ten for stability.

Around 10 people are using the solution in my team.

I rate Apache JMeter a nine out of ten for scalability.

Apache JMeter's initial setup is easy.

Apache JMeter's deployment takes half an hour to one hour.

Apache JMeter is a free, open-source solution.

I recommend users explore each and every functionality of Apache JMeter.

Overall, I rate Apache JMeter a nine out of ten.

We use this solution to load test functional behavior, and measure performance.

We really appreciate that the solution comes with a live community, which continuously provided plugins and support protocols.

We would like some reporting and analysis tools to be added to this solution.

We would also like the manual available for this solution to allow for better usability; it can be quite complex for new users, and the product is not always very intuitive. 

We have been working with this solution for 12 years.

We have found this solution to be a stable and reliable tool.

This solution is easily scalable, you can distribute the larger generation among several machines in the LAN, or you can use the cloud variation of the product.

This solution doesn't come with technical support, but there is plug-in wide community support, where organizations can ask questions about issues they are experiencing.

This solution isn't actually deployed, it is a zip file, which is extracted. It can also be deployed via public cloud, if an organization chooses the cloud-based model.

This is an open-source solution, so there are no licensing costs involved.

We would recommend that organizations considering this solution invest time in research of the product before implementing it. There are a lot of online resources that can assist with this, so the business can check if the solution will meet their requirements.

I would rate this solution an eight out of ten.

On-premises