What needs improvement with Apache JMeter?

Apache JMeter has room for improvement in handling larger infrastructures as it consumes a lot of CPU and memory. It requires integration with other tools for live metrics, which is time-consuming. Also, the failure response times are calculated in the overall response time analysis, which should be separate. Better script maintenance and integration with ALM or repository tools would be beneficial.

There is room for improvement in the scripting concepts. The scripting and even the results and reports were very elaborative and informative in LoadRunner, but not in JMeter because everything has to be done manually. And, whichever metric we need, we need to add it manually and start monitoring it, but not in LoadRunner. It was a very elaborate report. A lot more information, but not in JMeter.

To the best of my knowledge, the documentation could be improved. It should be enhanced to better support newcomers. Currently, only people with some experience can easily understand the content. Including more sample programs, applications, and use cases would help many more people adapt to using the solution.

Improving JMeter's sync time could be beneficial. For example, compared to a Hewlett Packard tool that required four machines for load generation, JMeter reduced this to possibly just two machines for the same workload.

JMeter is doing some good things with upcoming releases, but the main area for improvement is the extensions available. Another area of improvement is the reporting part, specifically regarding report generation. There are certain things like we can't merge custom metrics into the JMeter reports. We're limited to JMeter metrics, and other server metrics can't be integrated with JMeter dashboard. This forces us to rely on another tool. We should be able to add or custom-configure server details directly in JMeter reports.

Given that Apache JMeter is a free and open-source tool, documentation improvement may not be a major concern, as it is mostly contributed on a voluntary basis. The essential information is already available. However, in terms of the interface, there are occasional bugs, and the tool may not address them as quickly as some users would like. Fixing defects and bugs might take a considerable amount of time, with users sometimes having to wait for several months or even a year for the next release to address specific issues.

As features increase, it might become more complicated, and my goal has always been simplicity. Currently, I have to focus on other tasks, and I'm handling multiple responsibilities, so I can't juggle everything at once. However, if you ask me, I believe EJB covers most functionalities that are crucial. One improvement I'd suggest is adding a graphical aspect to the Gateway, making it a bit more colorful. Unlike JMeter, which lacks color, having a bit of color in the graphical aspects would be beneficial. Overall, for the essential features, EJB should work fine.

Apache JMeter could be a more user-friendly product from the end user's perspective. It requires someone with technical knowledge to administer it. This particular area needs improvement.

Apache JMeter's UI can be made more colorful.

The UI of the solution needs to be better. The UI takes up a lot of our bandwidth. So, we always run on the command line. Hence, improving the UI is needed. If it can be more lightweight, the editing can also be made easier.

There are some challenges in terms of recognizing the objects in some critical cases. These are object identifiers because Apache JMeter cannot recognize those dynamic objects.

Self-healing and page rendering for the end-users are not available in Apache JMeter. Hence, these are the two areas that need to be made available in future releases of the solution.

The UI has room for improvement. I would like to be able to measure web performance as well using the solution. Apache JMeter is only for infrastructure testing, and backend testing, but we cannot use it for performance testing because we need to do it through the browser.

The only thing is the learning curve. It's high. We'd like to see more third-party integrations that can be handled quickly. Support-wise, while the community is strong, it would be nice to have the option to reach out directly to JMeter. For performance testing, you need to correlate, et cetera, so we have to do it manually in order to get the right to regular expressions.

The reporting section of the solution can be better. Additionally, more plugins can be included in the next release.

The solution needs more metrics for reporting.

There are a lot of areas in this solution that can use improvement. The solution is not user-friendly, there is no framework for autocorrelation or parameterization.

The memory utilization in JMeter is very poor. The system gets hung up. I would like that to be improved. I'd like to see better reporting. The reporting offers limited information or details for a load test. So it would be highly appreciated if JMeter can provide better reporting and support multiple protocols. They could include a wide range of protocols, which LoadRunner already offers, while scripting.

What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses. As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected. What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter.

The solution's setup could be easier and security could be improved to minimize vulnerabilities.

We would like some reporting and analysis tools to be added to this solution. We would also like the manual available for this solution to allow for better usability; it can be quite complex for new users, and the product is not always very intuitive.

If JMeter could provide a web version of editing, that would be good. If JMeter can provide its own cloud version rather than depending on BlazeMeter, the commercial version of JMeter, that would be ideal. If we could have somebody right on the front end of JMeter, using it on any of the clouds, including AWS, GCP, or Azure, that would be very helpful. it would be better than me going for using commercial services. I would like to have some kind of cloud version that can be implemented. Or we would like a Docker version. A Docker version is something that I would look for.

Apache should have a graphic interface. That would help beginner users a lot. Sometimes it's hard to do what you need to do via the command line.

We would like more documentation to be provided for the advanced level features that are available in this solution, in order to improve development.

The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter. It's working on this, actually. It's an excellent plugin that you can use to record the scenario from Google Chrome, and it integrates easily into JMeter. They could also make it easier to generate the built-in report. Now, you run the tests and generate the charts in a separate column. The graphs and charts that display the test metrics could be better. I worked with another tool called Web Performance Tester, and its interface is better than JMeter's. They have intuitive graphs while you are running the tests, so you can see how things are going. It shows you the number of concurrent users logged into the system, the number of failures, response times, etc.

The reports in Apache JMeter could improve.

When you run tests with JMeter, it generates test version five, which is extremely large. Also, when you have a large number of tests to run, it requires a large size or memory size, which basically means it consumes a lot of memory. It would be helpful to come up with a way to be able to use Apache JMeter in a way where it did not use as much memory. It will be much easier, and beneficial for the individual to run it on their own machines rather than having a high-end infrastructure, more CPUs, or more memory that has been consumed by Apache JMeter.

It is not something that can be compared with Micro Focus LoadRunner. It gives the facility too easily; you do things through UI. With JMeter, you really do not have any easy UI to work as, like a Micro Focus LoadRunner. The stability could be a bit better. Compared to LoadRunner, it hasn't any proper UI. Recording the script is also not flexible in JMeter. In LoadRunner, we have a couple of options, such as URL-based recording and HTML-based recording. In JMeter, it's not like that. JMeter has a recorder, however, it is not easy to use. It is a bit tricky to configure the automatic recording in JMeter.

We use many plugins to customize our scripts, which is its main purpose. We wanted to be able to use a larger variety of customizable plugins to meet our needs. Along with our, JMeter, you would use a variety of plugins. The number of customization plugins should be increased. There could be improvements in terms of memory utilization. We are going to migrate away from JMeter in the near future. The data storage should be improved. Scalability could be improved. It should support more protocols.

Report generation needs to be improved. It is quite difficult to get to.

I am not satisfied with this solution. Its UI side is not so easy to use for beginners. It should be easier. Modeling a test is difficult. If you don't have much knowledge, you won't be able to do it easily. Testing APIs is also difficult. For JSON, you can use tools such as JSONBuddy, but they are difficult to get in JMeter. It should be easy to get JSONBuddy from the website. I have to use BlazeMeter only to get JSONBuddy.

I think it has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems when we are working in certain specific environments that have a proxy setup. When we want it to do a record with some new scripts, there are some challenges there.

Both scalability and stability could be improved in Apache JMeter. What I'd like to see in Apache JMeter in the future is ease of use in terms of scripting. A recording capability similar to what LoadRunner offers, where you can record scripts, make some modifications, then the script will be ready, is another advanced feature I'd like Apache JMeter to have. The two features would make it easier for new users to learn how to use Apache JMeter and help users utilize the tool more quickly.

I would like to see exceptions improved. The initial setup is complex and needs to be upgraded. It would be great to have additional protocols other than HTTP, HTTPS, and APIs.

In terms of platform support, they need to extend the support for backend platforms and more of the legacy types of platforms.

If JMeter could integrate with the EPM solution, it would be great. It could also be improved by offering more integrations for security. For example, most applications are secure with OpenID Connect protocols.

At present, if the number of virtual users increases beyond 10,000 when testing, then it results in a Java heap which causes the solution to crash.

JMeter's reporting is extremely rudimentary. The fundamental reporting mechanisms need to be drastically improved. It doesn't utilize an automatic session management mechanism or methods other tools use like parsing cookies and variables. Everything needs to be done manually. There's no automation.

The UI could be better. It can have some Reach UI also, which would be helpful, and maybe a relatively simpler way of using it. It needs simple modules. There are quite a lot of things which are kind of abandoned, so they can definitely improve on it. Integration with some of the other features should be managed. However, it's open source, so there is not much to complain about there. It's an open-source tool; we cannot ask for additional features really. The product could use some kind of filtering and monitoring and different degree of dashboards and analysis. If that can be provided, that would be very, very helpful.

The graphical user interface could be improved. In this tool, automation in general is almost non-existent. Everything is done manually. I would advise those who put this together to try to simplify it for their end users, such as being able to automate at their desks. Such as manual relations and social management. Purely on the feature set, it lacks automation, therefore it requires a lot of manual work.

It should be easier to combine multiple scripts. If you have multiple scripts, you need to write a new script to combine those scripts. The virtual user generator is slow.

Until now, JMeter is not supporting most of the protocols. It's widely using web HTTP and a few other protocols as well, however, it's not supporting the SAP or Citrix ones. Protocol-wise, the JMeter needs to improve. Recently, there was a Log4j error. They have since mitigated that, in JMeter, for the free version. The security concern was handled quite well compared to the previous versions.

JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases. A lot of the newer features don't work, and sometimes you have to spend a lot of time maintaining the scripts. That is something JMeter could probably look at. For example, in JMeter 5 they brought in a lot of new controllers. But there isn't a lot of documentation available on the Apache site on how you're supposed to use those controllers. They've explained the controller functionality, but there aren't any proper use cases to show that.

One of the drawbacks of JMeter is that it can't handle a large amount of load, which forces us to switch to other tools when we need to load more than a 5,000 or 10,000 user load. In the next release, I would like JMeter to be more compatible with other languages in the market.

Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports.

In Micro Focus LoadRunner we can go from the UI and we can configure it. There is no such feature in Apache JMeter. There should be UI-based recording history or logs.

If the solution was GUI based, I believe that it would be more versatile.

We're like the solution to be more user-friendly. As freeware, not everything is readily available. You can't play around with everything. That's just due to the fact that it's not a paid tool. When you pay for tools, you get a bit more. Not everything is supported by JMeter. It's limited. With JMeter, with banking encryption, we have struggled a lot. It's not as good as other paid tools that provide support and configuration capabilities that JMeter lacks. The solution doesn't really have good documentation, and, if you run into issues you can't simply raise a ticket. There's no help available to you. There are certain protocols that you can get on other solutions, such as LoadRunner, that you can't get on JMeter.

There are issues with the plug-ins which you need for reporting purposes as they make the reports quite heavy so you have to run them in non-GUI mode. If you go above the 200 user mark, the application creates a bottleneck and that's one of its major drawbacks. It means you have to run with a master-slave configuration with one system being the master, and multiple slave systems. It's not ideal and I think it could be simplified with a UI that provides direct configuration. In addition, the solution doesn't support SIP applications and some other protocols.

I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this. If I were going to be Mr. Selfish and say anything I want, I'd say a full feature GUI that lets me drag and drop different modules in line. It could have a simple-to-use GUI.

This is a difficult question to answer. On one side, JMeter is very flexible and allows for a high amount of customization. On the other, some tasks are common enough that it merits simplifying the process. Authentication for API testing could use improvement. Currently, it is a multi-step process to call, extract, and utilize a bearer token securely for API calls. This process is becoming a common enough task that a "wizard" for creating and consuming popular authentication models is merited.

The installation needs some work. It could be simplified. When compared with LoadRunner, LoadRunner is a more mature product.

They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available.

It should start supporting the presentation layer. It currently provides performance testing specifically at the application and API level. It can be extended to the presentation layer, which includes mainly Angular and React frameworks. It should also be easy to use and easy to train people.

The solution needs to improve reporting. Currently, there is not enough automation involved with the feature. For example, there should be an automatic way of saving reports. I have also found the recording should be improved too. When you are entering a launch in the controller the recording request should be inside it. Lastly, if they could make the technology better in terms of speed, this would help us.

The user interface could be improved. If they had better UI, it might make it easier to use. You really need a technical team in order to really utilize the product. The scalability could be better, or the process of scaling itself could be a bit more clear.

Currently, the integration pipeline is implemented by using Jenkins or a similar tool platform. These are continuous integration tools. As far as I know, integration is done by using custom scripts. It would be good if the integration with a continuous integration pipeline, like Jenkins or Hudson, can be done out of the box without using a script.

The reporting is not very good. When we run with multiple users, it takes a lot of memory. With respect to the recording and playback functionality, the auto-correlation parameterization is not easy and should be improved.

We have some scenarios for diameter load testing where TPS requirements are very high, 30K or 40K TPS. In the telco area, this is for simulating mobile usage. However, diameter load testing can be difficult in J Meter. The only way to imitate Diameter requests and process the responses of these requests is to implement them in the code of JSR223 samplers. JMeter generally provides synchronous calls. It's something that could maybe be improved in the future, because for achieving that very high kind of TPS, more than 30K, 40K requires a asynchronous solution. It's not a common thing, it's really very specific to the telco domain and a very few projects.

The GUI could be improved. When we go into GUI mode, there are occasions where it will not sync with our expectations. There are crashes that happen that will stop the solution from performing. It seems we get minor glitches when we go into GUI mode. The data client architecture that we have isn't so great. If we are to consume the data, it won't clear because there is tech running on different agents. When I need to pull the reports from different agents, it's not user-friendly. The reporting can be difficult to handle. It's hard to increase it if you are working on a client's architecture. It's not easy to get the data from one place or to do customizations. There are other solutions that allow users to model their load and structure with them. You can't do that on JMeter. On other solutions, like Silk Performer, you can do network packeting, which you can't do on JMeter. They should add this to the solution as a capability in the future. The support management needs improvement. Support is coming from consultants; you will not be able to get on-premise support from all of their agents in one place. On Silk Performer, for example, they have the capability where you can basically have a summarized report from different agents.

This solution should support the Ruby programming language for scripting. JMeter should support dynamic throughput so that we can reduce or increase it during the execution of the scripts. For performance testing, we would like to be able to select different bandwidths such as 3G or 4G. The interface could be made more user-friendly.

When we are testing with too many threads then the solution hangs. JMeter does not support JavaScript. Automation is difficult in JMeter.

Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like k6. It would be simpler that way. JMeter is a very old tool. It has been around for about 15 years. While it has been improved over the last few years, it is a little complicated to run several tests at the same time with different sites. JMeter could be easier. It would be a great improvement if it was easier to integrate with the CI deployments, with tools like Jenkins or CircleCI.

The solution is new to us. I'm not sure if we're using the full capabilities of the solution yet, but from what we have used, we're quite satisfied. In future releases, it would be helpful if there was an integration with ALM Octane.

They have to find a way to prepare the script or to prepare a detailed analysis. We have to collect all the information on each of the services we have to call. Based on this they have to collect in the phase of preparation for the performance test and then we can run our audit. It is easy to prepare a script to run a performance test. You can't rely on the support. It's something that is not fully working. The scalability of this solution needs some improvement. There is some work to be done with the integration.

The tool should be made a bit more robust, and better support should be made available.

RegEx Extractor needs improvement. It's a headache for many people. The solution could use some sort of educational features to offer tips and hints to help users navigate it better. They should improve the manuals and help files. I've searched the internet for answers over the past year or so, and I haven't come across anything that is helpful. The community help files are pretty good, but their own help files are not. In the next release, it would be helpful to offer more flexible graph handling to be able to combine different metrics into one graph.

The user interface is a little bit tricky.