We mainly use this product for load and endurance testing.
Senior Test Engineer at a computer software company with 201-500 employees
An open-source solution that offers good test script development functionality, but does not have a script exhibition feature
Pros and Cons
- "This solution is very user-friendly, and allows for a lot of data capture when testing."
- "We would like more documentation to be provided for the advanced level features that are available in this solution, in order to improve development."
What is our primary use case?
What is most valuable?
This solution is very user-friendly, and allows for a lot of data capture when testing.
We also found that the test scripts that the system automatically generates are extremely well created.
What needs improvement?
We would like more documentation to be provided for the advanced level features that are available in this solution, in order to improve development.
For how long have I used the solution?
We have been using this solution for two to three years.
Buyer's Guide
Apache JMeter
November 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.
What do I think about the stability of the solution?
We have found this solution to be very stable overall.
What do I think about the scalability of the solution?
We believe this to be a scalable solution, although we have not made any amendments in user numbers to date.
How was the initial setup?
The initial setup of this solution was very simple; the product is open-source, so only requires a download in order for the implementation to be executed.
What about the implementation team?
We implemented this solution using our in-house team.
What's my experience with pricing, setup cost, and licensing?
This is an open-source solution, so there is no requirement for licenses.
What other advice do I have?
We would recommend that any organization using this solution for test script development, may also want to look for a complementary product to handle script exhibition.
I would rate this solution a seven out of ten.
Which deployment model are you using for this solution?
Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
DevOps Engineer at a computer software company with 501-1,000 employees
Simple setup, reliable, and good online community available
Pros and Cons
- "Apache JMeter is stable."
- "In Micro Focus LoadRunner we can go from the UI and we can configure it. There is no such feature in Apache JMeter. There should be UI-based recording history or logs."
What needs improvement?
In Micro Focus LoadRunner we can go from the UI and we can configure it. There is no such feature in Apache JMeter. There should be UI-based recording history or logs.
For how long have I used the solution?
I have been using Apache JMeter for one year.
What do I think about the stability of the solution?
Apache JMeter is stable.
What do I think about the scalability of the solution?
I have found Apache JMeter to be scalable.
How are customer service and support?
Since Apache JMeter is open source there is no vendor support, you have to use the online communities available.
Which solution did I use previously and why did I switch?
I have used Micro Focus LoadRunner previously and it is better features than Apache JMeter. However, it is not free.
How was the initial setup?
The initial setup was easy for Apache JMeter.
What's my experience with pricing, setup cost, and licensing?
Apache JMeter is a free open source solution and it is why we are using it.
What other advice do I have?
For those who do not want to spend money on tools then this solution would be a good choice. This is the best solution in the open-source category.
I rate Apache JMeter an eight out of ten.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
Apache JMeter
November 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: November 2024.
816,406 professionals have used our research since 2012.
Script-generation is helpful for people with little programming knowledge
Pros and Cons
- "JMeter is easy to use for a user who doesn't have too much knowledge of programming or certain languages."
- "Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like T6."
What is our primary use case?
Our primary use for it is testing websites: load testing and performance testing.
What is most valuable?
JMeter is very easy for a user who doesn't have too much programming knowledge, as scripting is not mandatory nor essential to create the testing scenarios. This means the user doesn't need to know how to write code to run a complete set of load tests. Although, If you're an advanced user, JMeter does gives the possibility of using BeanShell, Groovy or Java to script your tests.
JMeter has a very friendly and intuitive User Interface which allows users to configure the scenarios reusing components already defined, which you can also modify to fit the requirements of the tests.
One very important feature of JMeter is that it is open source and it can be easily extended through a huge set of available plugins.
What needs improvement?
Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like k6. It would be simpler that way. JMeter is a very old tool. It has been around for about 15 years. While it has been improved over the last few years, it is a little complicated to run several tests at the same time with different sites.
JMeter could be easier. It would be a great improvement if it was easier to integrate with the CI deployments, with tools like Jenkins or CircleCI.
For how long have I used the solution?
Our teams have been using it for about five years.
What do I think about the stability of the solution?
It is a very stable tool.
What do I think about the scalability of the solution?
In my opinion, it is not scalable. We haven't found an easy way to do that.
We would like to increase our usage but we are trying to figure out the best way to integrate it with our new circuit. Our plan is to use a tool like this with customers. That is the reason we are evaluating the pros and cons of different tools. We want to incorporate these tools into our web development process, mainly in our CI/CD process. We are trying to figure out how to use tools that are designed to empower developers.
Which solution did I use previously and why did I switch?
We have used LoadUI and SmartBear with tools like Selenium. Some of the developers, at that time, were used to JMeter and they introduced it. It is a very nice tool.
How was the initial setup?
The initial setup is simple. It is not a hard tool to use. It has a very nice UI and the end-users are very comfortable using the tool. People deploy it themselves.
What other advice do I have?
I would definitely recommend JMeter. It is a very nice tool. It is easy to use and there are several components already developed in the tool that you can use to simplify the process of defining and redefining scripts. But I would like it to be more flexible and integrate with other tools.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Intermediate Technical Test Analyst (Mobile Lab SME HP Mobile Center and Appium) at a financial services firm with 5,001-10,000 employees
A scalable, stable solution that's very easy to use
Pros and Cons
- "The ease of use is the solution's most valuable feature. Also, the ability to easily create test cases is also very good. It's easy to just ramp up on the solution."
- "In future releases, it would be helpful if there was an integration with ALM Octane."
What is our primary use case?
We primarily use the solution for API testing.
What is most valuable?
The ease of use is the solution's most valuable feature. Also, the ability to easily create test cases is also very good. It's easy to just ramp up on the solution.
What needs improvement?
The solution is new to us. I'm not sure if we're using the full capabilities of the solution yet, but from what we have used, we're quite satisfied.
In future releases, it would be helpful if there was an integration with ALM Octane.
For how long have I used the solution?
I've been using the solution for six months.
What do I think about the stability of the solution?
The solution is stable.
What do I think about the scalability of the solution?
The solution is very scalable.
How are customer service and technical support?
We're not using too many features of the solution itself, so we haven't needed the assistance of technical support yet.
Which solution did I use previously and why did I switch?
We previously used LoadRunner. We decided to switch because on LoadRunner scaling up was a problem. We needed licenses to do so and it was becoming a problem for us to simulate the number of users we were expecting.
How was the initial setup?
The general setup of the solution is straightforward.
Which other solutions did I evaluate?
Before we chose JMeter, we looked at NeoLoad, but it looked to have the same issues surrounding licensing that LoadRunner did.
What other advice do I have?
We are using the on-premises deployment model.
The advice I would give to others would be to look at your requirements and decide if the solution is right for you. For us, it was API testing and JMeter is very good for this.
I'd rate this solution nine out of ten.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Founder & CEO at a tech services company with 51-200 employees
JMeter supports HTTP/HTTPS, DBs, FTP, and other system protocols and easy to crate basic scripts but has limitations on the reports
What is most valuable?
- Recording/Playback
- Templates for test plans (NEW)
- Integrate with Java and many other scripting languages
- Integrating with external plug-ins
How has it helped my organization?
- The recording feature makes it easy to create test plans
- They have templates that make starting very easy (this feature is added recently)
- Supports a lot of protocols (HTTP, HTTPS, FTP , SMTP etc)
- Simplicity of use
- As compared to commercial tools, it provides better reporting features through third-party plug-ins
- There's also a huge community around tool, and because of that when there's an issue, you can get solutions easily, and without cost For example, we dis a very complex testing of HL7 protocols, which JMeter doesn't support directly, but it's extendable to support the protocols
- Supports Java and other scripting languages and can extend capabilities.
What needs improvement?
Reporting could be better if it could be like commercial tools. For example, a nice chart and visualizations. You can’t rely on the first 100 test results because the ramp-up and ramp-down time gives false results with outliers. It would be great if an option to ignore first 100 and last 100 records from results and then generate reports, should be great. Needs to support more concurrent users by single JMeter instance (agent/slave)
For how long have I used the solution?
I've been using it for over last five years, and it is a major, active open source tool.
What was my experience with deployment of the solution?
No issues with recent releases
What do I think about the stability of the solution?
There are issues when tests are stopped with large number of concurrent users. Test is not stopped as expected.
How are customer service and technical support?
Customer Service:
There is a huge community around JMeter. Hence free support is available to the users.
Technical Support:Excellent (from the community)
Which solution did I use previously and why did I switch?
Grinder. Grinder was not user friendly and had to do lot of coding and low level configurations. It was lighter than JMeter.
How was the initial setup?
Initial setup is straight forward. You need to have have JDK or JRE for running JMeter.
What about the implementation team?
We have a team of testers capable of managing the JMeter perfromance testing project effectively.
What other advice do I have?
It's easy to use and free, a huge community, and it offers a risk free to start, so use it for application performance testing. It supports connectivies with databases, FTP, and other system protocols.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Engineer 2 at a computer software company with 10,001+ employees
User friendly interface, open-source and easy to install
Pros and Cons
- "It's very easy to install, and it's very easy to code and develop the script."
- "The memory utilization in JMeter is very poor."
What is our primary use case?
I use the solution for performance testing (Load testing, Volume testing and Endurance testing).
How has it helped my organization?
The solution has a friendly user interface and is a very good open-source solution for load testing. It supports both web and mobile network recording via proxy and offers a wide range of protocols. It has strong community support across multiple platforms.
What is most valuable?
The solution is open-source.
It is very easy to code and develop the script and it supports web and mobile scripting.
There is a lot of good documentation available.
The solution is stable.
What needs improvement?
The memory utilization in JMeter is poor. Limited in terms of scalability. The system gets hung up for complex scripts and heavy loads which requires improvement.
It needs improvement in analytics and statistics reporting. The reports offer limited information on a load test. I would highly appreciate it if JMeter can provide better reporting capabilities.
For how long have I used the solution?
I have been using this solution for two years.
What do I think about the stability of the solution?
It's a stable and reliable product. There are no major bugs or glitches and doesn't crash.
What do I think about the scalability of the solution?
The solution is scalable. However, it has a lot of memory issues during scale-up. That is the one concern. It's a bit limited in terms of scalability.
How are customer service and support?
I have never contacted Apache customer service for any issues. However, the documentation is readily available online and very useful.
How was the initial setup?
The initial setup is pretty straightforward and easy to implement. It's not overly complex.
I'd rate it a five out of five in terms of ease of implementation.
What about the implementation team?
I can do the implementation by myself.
What's my experience with pricing, setup cost, and licensing?
It is an open-source tool.
What other advice do I have?
I'd rate the solution eight out of ten.
Which deployment model are you using for this solution?
Private Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Sr Manager - Quality Engineering at a computer software company with 501-1,000 employees
Great performance testing for applications with a valuable load testing feature
Pros and Cons
- "We find the load testing feature valuable."
- "The reporting section of the solution can be better."
What is our primary use case?
Our primary use case for the solution is performance testing for applications, and we deploy it on the cloud and on-premises.
What is most valuable?
We find the load testing feature valuable.
What needs improvement?
The reporting section of the solution can be better. Additionally, more plugins can be included in the next release.
For how long have I used the solution?
We have been using the solution for approximately 15 years and are currently using version 5.3.
What do I think about the stability of the solution?
The solution is stable.
What do I think about the scalability of the solution?
The solution is scalable, but it all depends on the application. However, it is usually scalable by up to 2500 users. However, if you want to use it for any number of users, you can use JMeter as long as your machine supports it.
How are customer service and support?
We do not have experience with customer service and support because everything is stable.
Which solution did I use previously and why did I switch?
We previously used LoadRunner.
How was the initial setup?
The initial setup is straightforward. I rate it a nine out of ten.
What's my experience with pricing, setup cost, and licensing?
The tool is open source, so we do not pay for licensing.
What other advice do I have?
I rate the solution a nine out of ten. I advise new users to enhance their understanding of the solution via Youtube.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Software Testing Services Manager at Virgosol
Free and not difficult to set up, but it is difficult to test APIs and model a test
Pros and Cons
- "We are using it just for load testing. We are using its free version, and it is scalable."
- "Modeling a test is difficult. If you don't have much knowledge, you won't be able to do it easily. Testing APIs is also difficult."
What is our primary use case?
We are using it for load testing. We test other companies' programs.
What is most valuable?
We are using it just for load testing. We are using its free version, and it is scalable.
What needs improvement?
I am not satisfied with this solution. Its UI side is not so easy to use for beginners. It should be easier.
Modeling a test is difficult. If you don't have much knowledge, you won't be able to do it easily. Testing APIs is also difficult.
For JSON, you can use tools such as JSONBuddy, but they are difficult to get in JMeter. It should be easy to get JSONBuddy from the website. I have to use BlazeMeter only to get JSONBuddy.
For how long have I used the solution?
It has been about five months.
What do I think about the stability of the solution?
It is not so stable.
What do I think about the scalability of the solution?
It is scalable.
How are customer service and support?
Their support is fine. There are not many Apache documents, but you can find other documents on the web.
How was the initial setup?
It is not so difficult.
What's my experience with pricing, setup cost, and licensing?
We are using the free version.
What other advice do I have?
I would rate it a seven out of 10.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros
sharing their opinions.
Updated: November 2024
Popular Comparisons
Tricentis NeoLoad
OpenText LoadRunner Professional
Perfecto
BlazeMeter
OpenText LoadRunner Cloud
OpenText LoadRunner Enterprise
ReadyAPI
Oracle Application Testing Suite
Eggplant Performance
RadView WebLOAD
Akamai CloudTest
SmartBear LoadNinja
IBM Rational Test Workbench
HCL OneTest
Automai AppLoader
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros
sharing their opinions.
Quick Links
Learn More: Questions:
- Jmeter vs SOAtest?
- How does Postman compare with Apache JMeter?
- How does BlazeMeter compare with Apache JMeter?
- What is your experience with Citrix with JMeter?
- When evaluating Load Testing Tools, what aspect do you think is the most important to look for?
- SOAtest vs. SoapUI NG Pro?
- Does Compuware have a manual testing solution? Which manual testing solutions should we be considering?
- What are the top performance tools available to load test web applications?
- What is the best tool for mobile native performance testing on real devices?
- When evaluating Performance Testing Tools, what aspect do you think is the most important to look for?
Hello Janesh,
Thanks for this nice review.
Just one note, since JMeter 3.0 and even more in 3.1, JMeter has now :
- a new Web Reporting feature, see jmeter.apache.org
- Scalable Real-time Graphs in NON GUI mode thanks to 3rd party open sources like Graphite or InfluxDB, see jmeter.apache.org
Regards
Philippe M. from Apache JMeter Team