What is our primary use case?
We have a Neotys slave server configuration where we have one server that caters to three servers, and we test most of the load on Apache JMeter, particularly for a hundred users. We test the load for web applications, services, and the rest of the APIs, though our current setup for Apache JMeter isn't that big.
How has it helped my organization?
Initially, Apache JMeter had a complex configuration; its UI was tricky and required a lot of resources. Creating scripts and running tests on Apache JMeter was always confusing, but nowadays, with more documentation and UI enhancements, Apache JMeter has improved. Previously, recording and creating scripts was tricky, and you had to do it manually. Now there's a recording facility in Apache JMeter that lets you create and modify scripts and test faster, which helped improve my organization.
What is most valuable?
To me, what's most valuable in Apache JMeter is that it's a lightweight tool for application testing. It's the best load-testing tool for my company because Apache JMeter simulates your application during testing. Apache JMeter also creates threads with good server utilization. Apache JMeter allows you to focus on analyzing the situation, looking into measurements, response time, and client-server responses, which I find valuable.
What needs improvement?
Both scalability and stability could be improved in Apache JMeter.
What I'd like to see in Apache JMeter in the future is ease of use in terms of scripting. A recording capability similar to what LoadRunner offers, where you can record scripts, make some modifications, then the script will be ready, is another advanced feature I'd like Apache JMeter to have. The two features would make it easier for new users to learn how to use Apache JMeter and help users utilize the tool more quickly.
For how long have I used the solution?
I've been using Apache JMeter for more than six or seven years.
What do I think about the stability of the solution?
Apache JMeter isn't as stable because it sometimes crashes when you're running a test. The performance of Apache JMeter could be improved because testing on it isn't always as smooth sailing.
The tool is partially stable. You can't expect Apache JMeter to run well for enterprise-level, high-load applications. It's a good tool for more straightforward or lightweight web applications but not for CRM-type applications.
What do I think about the scalability of the solution?
Scalability-wise, Apache JMeter could be improved because if you try to implement it on multi-servers, the threads running on the tool don't hold up well.
Which solution did I use previously and why did I switch?
We used LoadRunner before using Apache JMeter. As Apache JMeter is open source, and we only needed to test lightweight applications, we were pretty sure we wanted to go with Apache JMeter.
How was the initial setup?
Apache JMeter is an open-source tool that you can install directly from the web with binary files, so setting it up on one to two machines is easy. The setup could be tricky if you hook Apache JMeter to three or more different machines, and it's also tricky when you execute it after.
What about the implementation team?
We implemented Apache JMeter in-house.
What was our ROI?
I've seen ROI from Apache JMeter, mainly because it doesn't cost much to maintain, and we can use it on a few lightweight applications.
What's my experience with pricing, setup cost, and licensing?
We didn't pay licensing fees for Apache JMeter because it's an open-source tool. We only paid for the machines where we installed Apache JMeter modules.
What other advice do I have?
I have experience with Apache JMeter, with version 5.5. as the most recent version I've used.
Apache JMeter is deployed on-premises, but my company did a POC with Apache JMeter and BlazeMeter. BlazeMeter is a CA proprietor tool where you can hook up Apache JMeter scripts. BlazeMeter is a cloud-based tool where you can run tests with the help of Apache JMeter scripts.
At the moment, only two people use Apache JMeter within my company. Two people can handle the deployment of Apache JMeter, while only one person is required to maintain it.
My advice to people looking into implementing Apache JMeter is to make the decision based on the application portfolio. For example, if it's more diverse, then using Apache JMeter could be tricky, but if you're only testing lightweight applications, Apache JMeter will be a viable solution.
Apache JMeter requires minimal investment, yet it has some returns, and it's a good tool, so I'm rating it as seven out of ten.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.