BlazeMeter Reviews

We use this solution as a tester. When it comes to 5G, there are loads of changes because we're trying to build the first 5G core network with the standalone architecture. Everything is based on APIs and API-based communications with a new HTTP/2 protocol. When we build the core network, we constantly change and tweak the network.

When it comes to testing, whether it's with Postman or any other tool, normally we run the test, make sure it works, and then move on. I was pretty impressed with [Runscope] because we can keep the test running 24/7 and are able to see feedback at any time.

A proper feedback loop is enabled through their graphical user interface. We can add loads of validation criteria. As a team, if we make changes and something fails on the core service, we can actually find it. 

For example, we had a security patch that was deployed on one of the components. [Runscope] immediately identified that the network mode failed at that API layer. The monitoring capability allows us to provide fast feedback. 

We can also trigger it with Jenkins Pipelines. We can integrate it into our DevOps quite easily, and they have webhooks. The validation criteria is quite simple. Most of the team love it and the stakeholders love the feedback loop as well. They can look at it, run it, and see what's happening.

The final solution will be across four different locations. The performance will run in a specific location. [Runscope] will run across different locations and test different development environments. At the moment, it's only on two environments. One is a sandbox where we experiment, and one is a real environment where we test the core network.

There are around 10 to 15 people using the application, but some of them only view the results. They're not always checking whether it works or not. We have multiple endpoints.

We use the solution on-premises.

The on-the-fly test data improved our testing productivity a lot. The new test data features changed how we test the applications because there are different things we can do. We can use mock data or real data. We can also build data based on different formats. 

For example, an IMEI number should be a 15 digit number. If you need various combinations of it, BlazeMeter can do it as long as we can provide regular expressions and say, "The numbers should be in this format." Mobile subscriber identities, which are pretty common in the telecom world, are easy. This solution has changed how we test things. Fundamentally, it helped us a lot.

Previously, most of the test projects that I delivered before I moved into automation used to take months. Now, the entire API test is completed within minutes. Because we look at millisecond latency, the tests don't take any longer. It's less than a minute. 

The moment those tests run on schedule, I don't really need to do anything. I just concentrate on what other tests I can add and what other areas I can think of. 

Recently, I have seen BlazeMeter's other products in their roadmap, and they're really cool products. They use some AI and machine learning to build new API level tests. I don't think it's available to the wider market yet, but there are some really cool features they're developing.

BlazeMeter reduced our test operating costs by quite a lot because normally to do the same level of testing, we need loads of resources, which are expensive. Contractors and specialists are expensive, and offshore is quite expensive. However we do it, we have to spend a lot of time. Running those tests manually, managing the data manually, and updating the data manually take a lot of time and effort. With this project, we definitely save a lot in costs, and we give confidence to the stakeholders.

For previous projects and even smaller projects, we used to charge 100k to 200K for testing. We're using BlazeMeter for massive programs, and the cost is a lot less.

This solution is a big strategic piece for us. We wanted to replace the legacy performance testing capability with BlazeMeter's performance testing capability. That was our first use case. Now, it's a more strategic platform with GUI testing, API testing, mock services, service virtualization capability, and test data capabilities. We aren't using everything at the moment, but that is the strategic intent.

We faced some challenges in getting multiple teams to adopt BlazeMeter. This was a big transformation for us. We used LoadRunner for 10 years, so changing to BlazeMeter was definitely a bit challenging. Organizational change management was involved. We were able to use other online resources to learn how to use BlazeMeter. There was resistance from some teams.

This solution is used by 24 teams across 13 divisions. There are about 75 engineers using BlazeMeter, but usage is higher. This is a hybrid solution.

The range of test tools that BlazeMeter provides is amazing. There are more than 18 tools, which gives us freedom of choice.

It's very important to us that BlazeMeter is a cloud-based and open-source testing platform. It's critical for our organization because we are increasingly moving from on-premises application hosting to cloud-native hosting.

BlazeMeter has definitely improved the productivity of our organization, especially because of the integration features. Engineering productivity has improved because people are able to use the tool of their choice. This increases our delivery velocity.

From the operational-benefits perspective, we save infrastructure costs because we don't have to host this massive product on infrastructure. We also save operational costs. We don't need a big team because it's a SaaS platform.

My organization has BlazeMeter licenses and we do all of our performance integration on BlazeMeter. We're not working on continuous integration. We are mostly working on an agile workflow. We don't have a CI/CD pipeline as of now. 

The largest test I have executed is with 6,000 users. It provides the advantage of using any tool workflow or LoadRunner performance center. It's the enterprise version. If you are using plain packaging meters, not BlazeMeter, you have to grow your instances, either on cloud or on-prem or one of those particular places. You have to do the setup and activities, monitor the activities, and do the housekeeping on the server. 

There is a repository of all the scripts that we have created. You can go back and compare tests to see what the tests looked like. If I want to compare something with whatever happened six months or one year back, I can do that. 

We do a lot of extensive testing on multiple products. For example, I'm part of the ServiceNow Platform. We can work with multiple products, multiple features, and multiple add-ons. So we require a lot of extensive testing activity and over the years, it has improved a lot. It has more protocols. It's catering to more technologies now. 

You can create pretty realistic scenarios by bringing multiple scripts into one test. You can make a realistic load.

The reporting and analytics capabilities helped identify performance bottlenecks. It has features where you can integrate JMeter into your plugins, which you can use to integrate with the test and you can do real-time monitoring. We have done that. Right now, within our team, we have a Grafana dashboard, which helps us do real-time monitoring of tests.

Our primary use case for BlazeMeter is performance testing. We leverage BlazeMeter as our enterprise performance testing platform. Multiple teams have access to it, and we execute all of our load tests with BlazeMeter and do all the reporting through it. We also use it for mock services.

We have a hybrid deployment model. The solution is hosted and maintained by BlazeMeter. We also have on-premise locations within our network that allow us to load test applications behind our corporate firewalls. That's for test environments and non-production applications that are not externally available. It's a hybrid role that is mostly SaaS, but the on-premises component allows us to execute those load tests and report the results back to the BlazeMeter SaaS solution.

The cloud provider is GCP. BlazeMeter also grants access to Azure and AWS locations which you can execute load tests from. They engaged with all three of the major cloud providers.

BlazeMeter gives us a centralized place to execute load tests, do reporting, and have different levels of user access control. BlazeMeter has a full API, which is the feature that's given us a lot of value. It allows us to integrate with BlazeMeter in our CI/CD pipelines, or any other fashion, using their APIs. It helps increase our speed of testing, our reporting, and our reporting consistency, and gives us a central repository for all of our tests, execution artifacts, and results.

BlazeMeter added a mock services portion. We used to leverage a different product for mock services, and now that's all done within BlazeMeter. Mock services help us tremendously with testing efforts and being able to mock out vendor calls or other downstream API calls that might impact our load testing efforts. We can very easily mock them out within the same platform that hosts our load tests. That's been a huge time saver and a great value add.

BlazeMeter absolutely helps bridge Agile and CoE teams. It gives us both options. BlazeMeter is designed so that we can grant access to whoever needs it. We can grant access to developers and anyone else on an Agile team. It allows us to shift left even farther than a traditional center of excellence approach would allow us.

It absolutely helps us implement shift-left testing. One of the biggest features of shifting left is BlazeMeter's full, open API. Regardless of the tools we're leveraging to build and deploy our applications, we can integrate them with BlazeMeter, whether that's Jenkins or some other pipeline technology. Because BlazeMeter has a full API, it lets us start tests, end tests, and edit tests. If we can name it, it can be done via the API. It tremendously helps us shift left, run tests on demand, and encode builds.

Overall, using BlazeMeter decreased our test cycle times, particularly because of the mock service availability and the ease with which we can stand out mock services, or in the case of an Agile approach, our development teams can stand out mock services to aid them in their testing. 

It's fast, and the ability to integrate with pipelines increases our velocity and allows us to test faster and get results back to the stakeholders even quicker than before.

The overall product is less costly than our past solutions, so we've absolutely saved money.

BlazeMeter is user-friendly and excellent. My company prefers the licensed version of BlazeMeter, which I occasionally use for scripting and designing on the CentOS site, specifically for creating real-world scenarios.

In LoadRunner, we currently cannot create realistic user load behavior as effectively.

I use BlazeMeter occasionally, depending on the project. It's not used for everything but when necessary, especially with JMeter, for specific testing scenarios. So, my use cases depend on and vary according to the project. 

In load testing, BlazeMeter is utilized to create realistic user behavior. It's particularly friendly for web LoadRunner users, providing proper pacing and timing control. 

Moreover, it allows for comprehensive control over user behavior across scripts, such as ramp-up and steady-state phases, which is crucial for conducting incremental load tests and more. This level of control isn't as easily achievable with other tools, but with JMeter and BlazeMeter, it's possible.

Incremental load testing is a type of test you can perform with JMeter. It's a technique that isn't straightforward without BlazeMeter. BlazeMeter facilitates this process.

I primarily use BlazeMeter for performance testing. I utilize the BlazeMeter plugin for recording, but not extensively for testing itself. 

I have recommended it to clients and used it once for a specific requirement when I needed to simulate higher loads.

One key advantage of using BlazeMeter is that it does not require me to manage my own infrastructure. It provides the ability to use hosts which are pre-deployed globally. Since BlazeMeter offers a higher limit on load simulation compared to standalone JMeter, it is beneficial for large-scale performance testing without infrastructure overhead. Moreover, some features like reporting are available out of the box in BlazeMeter.

We have a couple of use cases for BlazeMeter. One is performance testing. It allows us to aggregate the execution and reporting of our performance tests. We can also create automated functional tests relatively quickly compared to writing tests in a coded platform like Java.

Around 20 people in the QA department are using BlazeMeter to test Mendix- based applications. We're doing regression testing on 22 applications, and we have at least two environments that we interact with regularly: a development environment and a pre-production environment.

Before BlazeMeter, we didn't have a performance test aggregator. They were running one-off JMeter tests that weren't stored in a repository. JMeter can generate some reporting, but it's nowhere near as nice as what BlazeMeter provides. And it's more readily understood by the development teams that we work with and the management. That part is great.

We initially purchased the tool for performance testing, but we discovered that we had access to functional testing, so we started using that. That's been great for a lot of the same reasons. It increases visibility and gets everybody on the same page about which tests can run and the status of our regression and functional tests.

BlazeMeter can create test data for performance and functional testing. We don't have much use for that currently, but I could see that being useful for individual functional tests in the future. It's nice to have automatic data generation for test cases.

We haven't used BlazeMeter for shift-left testing. The functional testers embedded with the sprint teams don't do automation. That's all kicked down the road, and the automation is done outside of the sprint. While there is a desire to start attacking things that way, it never really got any traction.

I believe BlazeMeter has also reduced our test times, but I can't quantify that.
It's helped us with our test data challenges. I think they have a lot of great implementation, so I don't want to detract from that, but we have some problems with our applications and some custom things. I think we work on a different platform than many other people do, so it hasn't been as beneficial to us probably as it would be for many other people.

We use the solution for enterprise performance testing of various technologies including web services, APIs, and web GUIs.

We deployed the solution to increase our performance testing footprint, which we needed to upscale for the maturity of our operation. 

We have six on-prem load generators on our network, and the rest of our deployment is in the cloud. It's a very simple architectural design.

BlazeMeter opened up performance testing for us. Our old solution was a client-based performance testing tool, and for staff to access it, they needed to remotely connect to a Windows VM and book time with that controller. Now our tool is web-based, and we onboarded 12 to 14 teams to BlazeMeter, which would not have happened before. Our CoE team was the go-to for performance testing, but the solution has opened up the practice to the whole enterprise, making teams more self-sufficient, and that's the most significant benefit. Performance testing is no longer segregated to one team.