OpenText ALM / Quality Center Reviews

The overall task management. Managing all the assets and metrics.

I'm not familiar with all the changes, but they definitely have to be more DevOps friendly. They have to certainly be more open source friendly. That's the world we live in, where we can cut costs away from large-scale vendor contracts and service contracts. The ability to seamlessly integrate and provide more capability for those, managing those infrastructures and solutions, is going to be critical historically.

A lot of the vendor products - not just HPE or, in this case, Micro Focus, or whomever that I've dealt with over the years - were much more proprietary, much more exclusive. And what we're finding now is that the world doesn't work like that. Particularly as you move left and shift towards DevOps, application teams now don't consume from a central resource, they consume based upon decisions made internally to that application team.

Ultimately, what they need is flexibility. So any vendor product needs to have that intrinsic in its fiber, to be able to adopt open source, and integrate basically into almost anything, to expand out the choices available to an application; to make the decisions that need to be made independently at the time that they need to make them.

Not having looked at the latest, ALM Octane, just coming from the old world, at the time that it was necessary to implement a test management system to gather more information, metrics across different teams, different platforms, it served the purpose.

Things change constantly these days. There's a lot more going on. There are a lot more integrations available. I think if we're looking at the legacy owned product, I think its kind of come and gone as far as its ability to do what you need to do in a DevOps world. Any solutions in the future - I know ALM Octane is the heir apparent to the old infrastructure - it's going to have to be more DevOps friendly. It will need to be able to enable the consumers, the application's users who ultimately become the developers, to see the value in a more organized test management practice, versus more of a kind of hidden, under the sheets unit testing.

It's actually a whole trajectory of different solutions, different tests, that need to follow the pipeline for those folks. Anything that's not DevOps friendly, that's not DevOps easily consumable, to make the case for a more formal test management practice, is really going to end up by the wayside at the end of the day.

11 years.

My experience with the solution is that it has been fairly stable. What lies underneath is what creates the instability at the end of the day, the architecture that you are providing the solution on top of. I think once you figure out a viable, scalable approach to it, then the software itself, at least in my experience, has been very stable in running a test management operation.

It has met our needs. Just as long as you have the right architecture from the old days of physical server hardware to more of the newer stuff, which is VMware within datacenters - more virtualized.

And certainly the next rage for everybody is moving into Cloud infrastructure. So things are becoming much more self-service. You're getting model scaling. You're getting the things that are making the system more maintainable. But from a scalability standpoint, you want to be able to scale to the needs at the time that you need them. The Cloud certainly provides that capability.

I think like every company, they're changing the landscape. Support, in my experience, has been pretty good. There are always challenges based upon the routing/tier structure of who gets the issue first, how it gets routed, how it gets filtered down to the specific expertise that you need. That depends on your acumen as far as knowing your tooling, knowing your approach, what that's going to be.

Somebody who is very savvy, will obviously have frustrations coming into a tier-one support desk. Who they really need to go talk to at the end of the day may be somebody, and it will vary by company, like a tier-three, real low-level, very experienced resource support tech who fixes those issues. So it's going to vary based upon the customer's competency versus how they are routed through a support desk.

Testing is going to be testing. And the same challenges that you have in any of the different industries are going to be the challenges that you have in the ours, the insurance and financial industry, as well.

You know from DevOps to Agile, to Shift Left to Cloud, to managing your test assets efficiently and effectively, industry is really not going to make a difference.

I've been in a number of different sectors over the years. I've been in QA about 25 years, and having been in the natural gas industry, financials, insurance, HR systems. They are all pretty much the same challenges around testing. So I don't see a discrepancy in terms of the application you're testing. It's almost agnostic to the challenges that are innate with trying to test, within any type of development environment. Now, it just happens to be a more self-service DevOps model, where application teams make those decisions. But there's still always going to be those QA challenges.

All the modules that we have in ALM, one of them is the test module. No better tool in the market than ALM because the foundation is what you see, it's been in the market for so long. I really like the test module.

But it's not only limited to the test module. It is the entire application that's a management tool. So we use it for requirements as well. And the link is between your requirements and your test waves and test plans, and everything is in there. So it's a pretty good tool.

If you don't use any tool to manage your application people will - like some teams we have who use Microsoft Word documents to do their requirements, and Excel sheets to plan their test cases, and write the test case and then execute and store it. In the long run, that is not going to be helpful because this is a structured way of exhibiting your development. That is what had been missed.

So when we started using ALM in our organization - we'd been using QC for so long - when we finally started using ALM and we tied the requirements module to the testing module, that definitely benefited. It's because we can show a lot of data in there and now we can link to some 15 years of back data. Most of the applications are there from so long, so we still need to do the core functionality test. But we don't need to redesign and we don't need to search for Excel sheets. We know exactly who ran it, when they ran it, how the execution happened.

We do have some suggestions on reporting. Most of the time we need to download data and then we create reports ourselves. If there was a little bit better reporting available that would be great. The reporting is the one thing that we definitely want them to do more on.

Yes and no. Once in a while we'll have some bugs and they will fix them, but other than that it's pretty stable.

We have assessed ALM right now to be pretty stable. I don't see too many things that are missing in ALM right now.

Scalability is good. So far we have not had any issues with scalability. For the last three years we were using it as SaaS. Before that, for a while we had on-prem, but after moving to SaaS we have never had any problems. We run around 300 projects, we have about 100 projects which are light. We've got, at most, 100 users at any given time.

We've used it multiple times. One of the reasons is the SAP tab. There is this plug-in that connects with SAP, and whenever we do an upgrade or something we need to test with the SAP tab, and the software has been very helpful in doing that.

I already know the response that we get from support. We have a dedicated CSO who engages whenever we need something, when we sat we need this report, we need that data, then he will definitely immediately give us that.

No. It's been there about three years. I wasn't part of the team at that time.

If you are using ALM, you had best educate your users to use the entire solution, not only the testing module or not only requirements module, because you will have way more benefit using the entire tool. It is designed to supplement the entire lifecycle and will definitely improve your productivity and traceability. If you use bits and pieces of the tool then the whole intention of developing the tool is not fully utilized. So use entire module, all the modules in ALM.

ALM: You cannot just say one feature is most important. You get the most value
using all modules from Management to Defects. When you use the tool end-to-
end, you can pull efficient project reports (especially scorecards) from the
Dashboard. So everything is integrated and only then you can evaluate the tool
fairly. ALM is very flexible and each module can be used independently, but
when you do that you are only using the tool as storage, not as a test
management tool.

UFT: It became much more stable tool in terms of object recognition over the
years. It is easy to use as long as the user has basic software development
knowledge and understands that the software automation process is not just a
record/playback.

ALM: We currently successfully manage all testing projects due to ALM’s invaluable capabilities, which are listed below:

• Built on best practices with a flexible structure, organization, and documentation for all phases of the application testing process.
• Serves as a central repository for all testing assets and provides a clear foundation for the entire testing process.
• Establishes seamless integration and smooth information flow from one stage of the testing process to the next.
• Supports the analysis of test data and coverage statistics, to provide a clear picture of an application’s accuracy and quality at each point in its life-cycle.
• Supports communication and collaboration among distributed testing teams.
• Reduces time needed to create test execution summary reports.
• Reduces the time needed to write and execute manual tests with HPE Sprinter tool.
• Users can capture their actions automatically as steps in a formal test.

UFT: We save time executing smoke and regression tests. We also use UFT to create test data.

I would like to see better Reporting functionality especially more sophisticated graphs, for example Actual vs. Planned or high level progress graphs using indicators like traffic lights. I would like to see more sophisticated or flexible Dashboard views, such as editing and resizing. I use scorecards and pull them into the Project Reports using customized templates. Scorecards can only be refreshed from the Site Admin, which then test leads has to depend on the ALM Admin to refresh the reports if needed after the scheduled auto run. There should be ability to refresh scorecards (execute KPIs) from the project itself or give more frequent auto refresh option like even every 5 min. This is a really burden on the team.

I would like to see Requirements mapped to test steps so we can combine multiple requirements validation in to one test case but map the verification steps to the associated requirements, so if the step fails only fails one requirement not all. When we are operating in an Agile world we do not have time to write test cases to capture one-to-one coverage. I know ALM allows many-to-many mapping but we cannot get true requirement pass/fail status if we use many-to-many option. Test configuration option kind of on the right path, but can only be use for data driven test cases, I cannot add design steps. If we can add design steps to a subset of a main test using Test Configuration option, I think we may be able to differentiate individual requirement that was failed without failing all the requirements mapped to the main test case.

We have used this solution for 17 years.

I did not encounter any issues with stability.

I did not encounter any issues with scalability.

In terms of technical support, I usually get solutions to my issues. I did not have any issues to call technical support for a long time.

If you follow the instruction, the setup is straightforward. It definitely requires an experience user to do the installation and setups, especially for upgrades.

I always used ALM and UFT. However, I had training and evaluated IBM JAZZ tools.

HP ALM is a good tool for a centralized and coordinated view of requirements, tests, defects, and iterations.

The main barriers of entry are cost and implementation, especially if an enterprise implementation is the best solution

Being able to have one place to review defects, testing progress, and defects was very useful.

Merging 40 different streams, just for defects, into one solution that had good search and reporting capabilities saved a significant amount of time in coordination, defect management, and by consequence, there was better control of the quality of delivered software.

The main barriers of entry are cost and implementation, especially if an enterprise implementation is the best solution

I have used this solution for eight years in a variety of versions and companies.

The biggest challenge was finding the appropriate resource balancing for the enterprise release. It is not very clear how that was going to be implemented due to documentation in 2010.

If there is a need for third-party integration, the documentation is not very good. We were able to integrate with FIT, but it took a very capable programmer to figure out how to do it. Again, this was in 2010. Hopefully, the documentation has improved.

I did not encounter any issues with stability.

I did not encounter any issues with scalability.

Customer service was adequate.

Technical support was adequate

We used a lot of home grown 'tools' and spreadsheets in one location and Lotus Notes in another.

In one instance, it was straightforward because anything was better than spreadsheets.

In the location that used Lotus Notes, there was a significant amount of resistance because of loss of control.

Neither instance was due to the tool, but it was due to cultural issues.

The implementation was done in-house.

At a Fortune 100 company, we achieved a reduction of 30% of defects in the first year and decreasing percentages the subsequent years.

The dollar figures were proprietary, but were significant even for an $11 billion dollar revenue company.

The main concern is that there is a significant dollar investment, so do good research to make sure the tool will meet your needs.

We evaluated IBM tools, as well and a couple of Open Source tools.

Being able to manage tests as this is something very difficult to find in other products. There are a few open source ones that handle test management, but right now HPE ALM is still the best solution to handle tests.

It helps us to keep track of everything happening. When you test the software you've got results. Results can be OK or not OK. If you just get the results in Excel or things like that, you cannot work as a team because just one person at a time will be able to access it. With ALM, we can have several people working on the same product at the same time. Then we use it a lot for trustability, so we can add trustability to the facts, to requirements. It's very useful for that to verify everything that happens.

As soon as it's available on-premises we want to move to ALM Octane as it's mainly web based, has the capability to work with major tests, and integrates with Jenkins for continuous integration. This is lacking in the standard ALM which was great a few years ago but it did not evolve enough, and that's why we are waiting for Octane.

We've used it more than 15 years, so it's very stable. There is a new version coming, ALM Octane. Octane is new so we don't use it yet.

We have plenty of projects with the current ALM, so it scales well. I'm not afraid of an issue with Octane and believe it will be the same.

I'm disappointed with the support as they're not reactive enough. They don't know the product very well, and to have things changed we need to access R&D directly and then things move. Otherwise, it's kind of difficult.

Beforehand we were using just paper and Excel, and things like that. As soon as ALM was tested at the time we began to use it and sensed it's presence in the company and now every tester is using it.

For ALM it was complex because it's not fully web based so you need to install a client on your desktop and with all the Windows security stuff you need to be an admin on your desktop so it's a very complex set up. On the service side it's kind of complex but we have tech experienced people to do that and to set up the database and everything, so it's OK. With Octane it should be really much simpler because for the user because it's just a web application so you've got nothing to do.

I'd rate the pricing as 3/10 as it's very expensive.

The first criteria we look at is functionality. We have plenty of different projects so we need a full spectrum of functionality. The problem we have today is the price. It's a very good solution but it's expense so we are challenged by our finals and everything but the price.

If you have the money then you can go with ALM, as it's a very good product. You won't have any surprises with it so that's good. Otherwise, there are some open source solutions that are a little bit less functional, but you can play with them and get them to work, products like Squash TM or things like that.

The most valuable thing is the flexibility of the customized options. That makes it more powerful than any other tool. We can customize based on the project and on how we want to control the testing.

We used to have 10 different Excel spreadsheets for one project. Then, we switched everything: paper, Excel, etc. to be done in ALM. There is no outside noise and everything is done under one umbrella.

The canned report site could be improved. You can get your report but you have to do some stuff. If the project doesn't have a good, strong user, they don't get these reports. If we have more canned reports from the ALM site, this will solve some issues.

We have been using the ALM call center since the Mercury times, so the last 10 to 12 years.

It's very stable. In last 12 years, we've probably had two/three downtimes. But, nothing concerning their application.

Yeah, it is scalable. 10 years back, we started with five users. Now, we have 38 confirmed licenses. Over the years, we have grown from having just a few projects to having more than 25 large projects.

Our experience with the HPE support was not great. We have not used Micro Focus yet. Based on that, we switch to a consulting firm, Melillo, for the support because we were not getting direct answer from the HPE support, therefore we switched because of that. Now, we get a better service. Hopefully, with Micro Focus, it will be better.

If someone is doing the setup for the first time, it might be a little complex for them. However, if you are continuously upgrading, then it should be fine, because all of our upgrades we have done in-house. We never went to a company to get that bit done. If you plan it right, you can have the upgrade very smoothly done, so the user isn't affected.

Most important criteria when selecting a vendor: support and stability of the product. These are the two most important things to us. We want to have continuous improvement, because there are places to improve; we also don't want rapid changes, because they do affect the user, so that balance is important.

The Business Process Testing module and approach to testing in QC is its most valuable feature.

For manual test cases, we need to write test case each time and if any update or CR comes then we need to go to each test case and update, which is very time consuming. But, with BPT we can update it in less time as would otherwise take to update two or three business components. After a refresh, it will automatically update the whole test set, which is over 100 test cases.

I would like to see a bit of improvement in its look and feel.

We've used it for seven years.

There were issues with the deployment.

We had some issues with the stability.

We had some issues with the scalability.

8/10

8/10

Earlier I used Mantis, but it was not user friendly and had no functionality apart from defect tracking. But HP QC is defect tracking by default. Test Case Execution tracking and reporting functionality which will serve all purposes for testing processes.

The initial setup was not complex.

We used a vendor team with in-house machines for the implementation.

For testing processes and improvements, I would suggest you use this product. But, if you're looking at cost, then that might be a concern, but no doubt it is the best tool for testing.

This is managed by Tieto, our managed service testing partner. We use ALM as a repository for our automated test scripts. This is only the very beginning of the of our testing and managed testing service journey. The reason we use ALM is it's ability to schedule tests and nightly runs. It creates reports and statistics.

We are only starting off now. I'm able to present the progress on our work with the test-information initiative. I can keep a close eye on what's going on to monitor the progress and to schedule the test runs.

I used Quality Center 5-10 years ago, and I had no issues with it. It is also the de facto industry standard of test management tools. I don’t have enough insight at this point in time. If you ask me in half a year's time, I'm sure I'll have loads more information.

We have had no stablity issues so far.

I know for a fact that it's possible to scale it up. We might add another test management tool in-house that's been there for a long time called Rec Test; a Swedish solution.

It’s a very simple test management and requirement tool. But in the long run, ALM will probably support us better, so that is on my radar to keep track of and see how we can implement them better. This will take a lot of training and convincing of end-users.

I have not used technical support myself, but Tieto is actually doing that in an effort to improve their own framework and initiate a closer relationship with HPE.

Setup was a very smoothly written, spot-on theater.

Not formally, but informally through my own experience. Our use with Rec Test, as well certainly requires more training. It requires a more structured way of working. You really need to set up a good structure, and make sure everyone is following that structure. Otherwise you'll have a mess in no time.