OpenText ALM / Quality Center Reviews

We use Quality Center to track functional testing and record automation testing scenarios results. There are around 1,000 users at my company. 

ALM Quality Center is a reliable, consolidated product. 

Quality Center's UI is outdated, and it's a little bit slow on the login part and different parts of the application. That's why we're switching solutions. I believe most companies are switching to Octane or something else. Micro Focus should enhance the interface and reports.

I've been using ALM Quality Center for seven years.

ALM Quality Center doesn't break down too much.

ALM Quality Center is scalable. There isn't much impact on performance when you add users.

Setting up ALM Quality Center is easy. It's not complicated to set up the on-premises solution.

I would rate ALM Quality Center six out of 10.

On-premises

The primary use case is test management, e.g., test executions using UFT combined with Business Process Testing. We do also requirement traceability, where we pull requirements out of a source system, then we link test cases to those requirements in order to have a coverage matrix.

It empowers us to do more testing. Our testing is being done for customers. 

The solution enables us to conduct risk-based testing. We link this solution to requirements of a certain risk factor. Once it's covered at least one time, it will show us in a report that it has been covered. Most tests are running automatically with UFT, so the check is already there in the automation, and there's no impact to us.

The Test Plan feature is the most valuable because of the test execution.

Security is covered. HTTPS works well. There is also support for LDAP over SSL. Those are the most important security features.

Within Quality Center, you have the dashboard where you can monitor your progress over different entities. You can build your own SQL query segments, and all that data is there in the system, then you can make a dashboard report. That works fine.

Managing multiple projects is possible when you have the full ALM license. However, we have the Quality Center license, which can be managed poorly. This is because you cannot look or report across projects.

We don't use Single Sign-On because this is available from version. Therefore, we do not use it right now. Also, it needs to be tested and we haven't tested it yet. With test automation. If you have Single Sign-On and want to make use of another user, that can be challenging. It is good for normal users to use Single Sign-On. However, it's not really a must at the moment, though it is good that the solution finally supports SSO.

Making Quality Center available to connect to external tools is doable, but it takes some work. With our current version, it is not fit for external entities. Connecting to external entities is easier to work with and report in using the newer versions. However, if you really want to use other tools, I would suggest giving ALM Octane a try.

The defect management module has room for improvement. E.g., for Jira tickets in defect management, they could have a direct link with Jira. However, with Micro Focus Connect, you can set up a link between Jira and Quality Center. 

Requirements management could be improved as the use is very limited. E.g., they have always stated that, "You can monitor and create requirements," but it has its limitations. That's why companies will choose another requirements management solution and import data from that source system into Quality Center. Micro Focus has also invested in an adapter between Dimensions RM and ALM via Micro Focus Connect. However, I see room for improvements in this rather outdated tool. I feel what Micro Focus did is say, "Our strategy is not to improve these parts within the tool itself, but search for supported integrations within our own tool set." This has not been helpful. 

I want to see Atlassian as part of the ALM solution. ALM Quality Center is more from a waterfall approach where Atlassian has already evolved into more of the DevOps and agile part.

I started using Quality Center ALM with version 9.2.

I just engaged with my new customer to do an upgrade. At the moment, it has been stable on all versions of Quality Center. However, I'm quite positive that will room for improvement will be needed shortly after we release the newest version of Quality Center.

Do not wait too long to upgrade. The longer you wait, the harder it gets to upgrade to the latest version with the newest features. Just like buying a car: You do not buy a car, then not go to service.

It is scalable in terms of high availability when you add an additional node because it's licensed for ALM. For Quality Center, this makes it less scalable. However, this is the perception from the vendor that the Quality Center addition is not for big enterprise. It's for a corporation, but not for an enterprise. Normally it's for bigger companies: 2000-plus users with over 1000 projects and domains. Then, they need to scale up with additional nodes, which will make it scalable enough for ALM.

It very much depends on the support engineer that you get. In the past, I've noticed that some really do not know the tool. Sometimes, I challenge first line of support or can come up with a solution faster than the support, but that's because I've also provided technical support for ALM in the past on the behalf of HPE. I know a bit more than the normal user.

Sometimes the support is very good, and sometimes it's a bit poor. E.g., if you go to the second or third of line support engineers, they really know the product. I've also worked with R&D in the past, and that goes beautifully. 

The installation is quite straightforward. Then, the implementation is based on one project, so it cannot go wrong.  This is for a very quick start. You will need more skilled people in your projects for implementation if you want reporting, traceability between requirement tests and defects, and release management. 

I always see ALM as an enterprise solution, so I don't go for the project implementation. You also need to maintain it. If one project has an issue, it may be very different in another project. There's also an issue when you have a user who is working multiple projects. E.g., where does the user have an issue? From a maintenance perspective, project implementation is not very handy so I always try to treat it as an enterprise solution, not as a project solution.

Testing time has decreased for manual execution because tests are being executed with UFT.

ROI is very difficult to say. If you don't test, you don't know how good or poor your quality is, but effective testing always costs money. However, it is very important for your return investment to know the value of your tests. What I've seen until now is that it's not being monitored that much. We have this tool because we need to test and prove the quality of the tests that we have been doing, but there will always be bugs and defects in production.

The solution has the ability to handle a large number of projects and users in an enterprise environment with the correct license.

Most vendors offer the same pricing, though some vendors offer a cheaper price for their cloud/SaaS solution versus their on-premise. However, cloud/SaaS solutions result in a loss of freedom. E.g., if you want to make a change, most of the time it needs to be validated by the vendor, then you're being charged an addition fee. Sometimes, even if you are rejected, you are charged because it's a risk to the entire environment.

With IBM Rational Quality Manager, you need to stick to the rough process and first train your end user versus ALM Quality Center's basic features, which are very easy to understand.

Make sure you have your build requirements and which features are important. Are you running projects for DevOps, agile, etc.? Also, make sure that you can evolve your tooling and not stay on the same tooling for years, knowing that your business users grow faster and have different needs.

Micro Focus does invest enough, but most investments are now going towards ALM Octane. I've seen that they are investing in adapters where you can say, "We're going to migrate from ALM.net to ALM Octane," if not entirely, then partially. There will always be projects in ALM.net, and they will keep maintaining ALM.net because there are many customers on it. Customers do need to realize that IT is changing and that you need to modernize as well.

I would rate this solution as an eight (out of 10), though I would rate it less for DevOp/agile.

On-premises

Test management and reporting. Those are the two most important things. I tell my customers that the two main reasons they have ALM:

• Quality management, project management from a QA perspective - testing, defect management, how testing relates back to requirements.
• And reporting to make good business decisions in the future.

When I was a customer, it improved my organization because I was able to manage, to enforce standards on building tests, executing tests, and manage centralized reporting.

Now, I translate that over to my customers from various levels of the spectrum from complete, "We have no idea what to do to, we're doing stuff but we know we need to change," to "We've got some stuff and we just want to tweak what we're doing now."

I'd like to see the idea of users being flushed out more, so not just, "This defect is now assigned to a particular person," or "This person is assigned to execute a test."

I want to see the users expanded out to teams where you have five users and the sixth user is the manager, so the manager can roll the idea of somebody being responsible and accountable. The idea of things being assigned to a team of users and users belonging to that team. There are ways of getting around this in the tool because it's very customizable, but I'd like to see that separate from the idea of using security groups, which is one way of getting around that.

I'd like to see the concept of teams put into it.

ALM has gotten more stable over the years. It's a stable app. Like any other large, complex application, you run into things every now and again. We have a system to report things and get them taken care of.

I have customers that are small and customers that are enterprise-wide. So I'm able to deploy it in both kinds of environments and customize the tool, depending on size and level of maturity, for any kind of customer. Also within any vertical as well.

I have used tech support. Mostly because I'm with a consulting company and we also support ALM. We have our own internal support organization that people can get into.

In terms of Micro Focus support, because I'm a more advanced user - I've been using this tool since version 7 - I typically don't get a whole lot from first-level support. I tend to want to go right up to second, third, or even directly with the development organization. So I'm more the outlier, edge-case kind of person compared to most customers out there.

Once I get to the people that are at the level that I know I need to deal with, they're good. I'm also dealing with the people on the other side of the ocean, working directly with people who may have actually coded ALM to begin with.

When I became a customer in 2000/2001, when I first started, I was involved in the decision to purchase the solution. Now, as a professional services consultant, that decision has been made and I'm going in there to either deploy, upgrade, or help them use ALM to best suit their needs. In some cases I help them figure out what it is they need to have ALM do for them or how to customize it best.

When I was a customer, we were not using another solution. We were completely manual and I was a department of one. I was the QA organization for a small development company and the two company owners said to me, "We want to invest in this, go look and see what's out there and show us what our options are and what you think the best option is."

What caused us to switch to this solution was the customizability. The fact that we could make it give us the information that we needed to get out of it. The support organization seemed very top-notch. I actually learned a lot from the support organization when I was getting started in it. And I found it more intuitive then the Rational solution.

I've deployed it in many organizations because I'm a consultant. I've deployed it, upgraded it, customized it, in various ways for different customers.

In terms of complexity, it really depends on the needs of the customer.

When I was a customer in a small development organization that only had 20 people in the entire company, I deployed it, I did the customization - that was way back in the day.

Now, I have customers along the entire spectrum from small to large enterprise. Some customers are okay with near vanilla, out of the box. And some customers have very complex sets of business logic that they feel, for whatever reason, need to be enforced as far as how their defect management lifecycle is going to go. How their test construction, test execution lifecycle is going to go, how they want to manage requirements, and that can require significant customization.

Some of my customers have compliance concerns, they have digital signatures and they have FDA 21 CFR Part 11 compliance. They have all of these rules that they have to follow and some of them are subject to interpretation, so with one particular rule I have one customer who says, "This is how we interpret the rule," and they have me customize it one way; and I have another customer who says, "No, we're not going to interpret it that, way we interpret this way," and it's a completely different set of customizations.

Back then it was Mercury Test Director, which is now ALM. We were also looking at the Silk products, and we were looking at the Rational, now IBM, products.

When selecting a vendor to work with, I want to see that the technical people are really knowledgeable of what they're talking about. I want to know that the tool can give me what I need, not just, this is a standard proof of concept. I want to see what I need to see, and I want to know that, down the road, I'll be able to either get out of it what I need or be able to learn or have somebody come in to help me get out of it what I need. Because if I'm not getting out of it what I need, then I've wasted my money.

I give it a nine because nothing is perfect, there's always room for improvement, especially when you're talking about an app system as large as ALM is. I've been using it for so long it's kind of second nature for me to think about where its strengths are, and know that if I can't get something done one way there's always another way around it. Or I can integrate something into it or build work flow to make the UI behave the way I want it to.

Regarding advice to a colleague about ALM, remember that your process and your methodology should be driving what you need out of their tool and not the other way around. Tools can do some really cool stuff. You may look at it and say, "Okay, maybe we could get some value out of this feature that we're not doing today." But don't make that the driving force. It really needs to be able to support what you're doing and force the things that you want to get out of it. Because there's a truism in reporting: If you don't capture the data you can't build a report that's meaningful. So make sure it can get you what you need.

• Requirements sync and traceability: This allows us to see how many requirements have been tested and to show auditors this information easily.
• Test Execution (Test Lab): This allows us to track our manual tests with date and time and enter actual results and screenshots.

QC has been invaluable in the past for documenting our testing process, especially when needed for audits.

The Active-X technology requires client-side installations that are difficult to manage in environments where the tester's PCs are locked down to prevent installs. Test management is too rigidly dedicated to older ways of testing with scripted test cases. More support for newer approaches, such as exploratory testing or behavior driven testing would make QC more relevant to the way testing is done in many current contexts.

ALM requires that you install client side components. If your organization does not allow admin rights on your local machine, this means you will need someone to run the installation for you with admin rights. This client side install is also limited to Internet Explorer and does not support any other browsers.

As far as the test structure goes, you are limited to to a step-by-step test case with description, expected result, and actual result for each step by default. This makes it difficult to support an exploratory testing approach with ALM. Of course, much of this part of the tool can be customized, but it still pales in comparison to something like the Test and Feedback tool that Microsoft provides for exploratory testing.

My understanding is that the newer Agile Manager product is more friendly to exploratory approaches, but I have not used this product yet.

We have been using this solution for 16 years.

In terms of stability, the QC client crashes often when attempting to expand a node on a tree. Upgrades are a nightmare and documentation is difficult to understand.

There were no issues with scalability, but I have never managed a large user base.

Technical support has gotten better than it was a few years ago, but Tier-1 seems to just go through the motions of asking questions I've already answered.

I have used other solutions, but many do not have the traceability requirements that ALM does.

The initial setup was highly complex, mostly because of the database setup. Upgrades are even worse, especially if you need to migrate to a new server, since the repository needs to be copied over.

Make sure you get the correct license for your needs. The full ALM license lets you use the requirements tab, along with test automation and the Performance Center. You can also just buy the Quality Center edition (Manual testing only), or the Performance Center version (Performance Testing only). I have no idea where they get their pricing numbers from, but they seem to always be negotiable.

I have used Oracle's Test Management solution, Zephyr, Bugzilla, TestTrack, JIRA, and others.

Be sure to have a DBA available when you install. There have almost always been changes needed to the DB when I have installed the application.

The Open Test Architecture (OTA) and development of the REST API. The OTA is a published set of functions that administrators and users can use to interact with HP ALM programmatically. The most common example HP ALM users would recognize is the Microsoft Excel upload template, which allows users to upload test scripts to HP ALM projects directly from an Excel worksheet.

The REST API sneaked into HP ALM with little fanfare. The REST API has no application overhead and is fast. HP extended the API through patches in v11.0. Please check your current version and patch level to see which functionality is now included in the REST API.

On a recent multi-year project, the average defect resolution time for all defects was over twenty-two days. My goal was to reduce this number by 20%. It was an easy goal to reach because no one realized that 80% of 22 days was still a number out of bounds for defect resolution. I used custom fields, defect workflow and custom reports to move defects through their lifecycle. Within thirty days the defect resolution time was reduced to 3.1 days and averaged 1.1 days over the next eighteen (18) months.

The graphical user interface has the most room for improvement. Not all screens within the integrated suite refresh the same, some screens or activities are self-refreshing and some are not.

I would also like to see the “Disable Quick Runs” added back as a site parameter or built as an internal function within a project.

I've used it for seventeen years (1998 to present). This product was initially developed by Mercury-Interactive and released as Test Director. My first enterprise installation and administration experience with Test Director 2.0 was in 1998.

Over a seventeen year period, yes. The key to maintaining a site today is in patch management. Keep the patches up to date.

When the patches lag too much, it may be safer to build a new site and port the data than to try and patch an existing site in place.

Customer Service:

Customer support for this product is with the vendor, not necessarily HP. After a site has been up and running a few years and all the original implementers are gone, it can take some time to even determine the vendor. My satisfaction level with vendors range from acceptable to excellent.

Technical Support:

My personal satisfaction level with HP service and support website is low. I get the majority of my technical information from colleagues or third party discussion forums.

In the Application Lifecycle Management space, HP ALM and IBM Rational are the two big players. I recently participated in an evaluation of the IBM Rational Jazz Platform. The client had been using IBM’s ClearCase and ClearQuest for many years. During the evaluation, an unrelated IBM audit detected a long dormant pack of five ClearCase licenses on an active server. The cost associated with this incident ended our evaluation of the IBM solution.

Historically, most people considered this to be a defect tracking only tool. In that domain, tools are plentiful. Over the years I’ve used VI editor on UNIX, Microsoft Excel worksheets Microsoft Access databases, Bugzilla and Notepad for defect management.

The installation and configuration of an HP ALM site is straightforward for those with enterprise software deployment experience. An installation requires at minimum, a dedicated server with an operating system and database connection. The most typical, physical or virtual, hardware configuration I encountered over the years was a single Microsoft Windows server running web, license and application software servers. Both Oracle and MS SQL Server databases respond adequately, and when given a choice now select a schema based on DBA agreeability.

The installation of an HP ALM site establishes a service endpoint for communicating with other applications via Web Services. I believe the configuration and management of these services is the most complex part of a site installation and requires substantial planning to map fields and permissions across multiple applications.

HP ALM resellers typically perform the initial set up and configuration of the HP ALM site and user projects. In some cases, larger testing firms are also resellers and provide the tool as part of the project. I fundamentally disagree with buying a tool from its eventual user.

I advise clients to do the upfront planning and limit users with access to the site administration console to three or less. The planning required for a successful implementation requires much more time and effort than the deployment itself. Deployments are typically scripted while planning requires humans. Access to the HP ALM site console is separate from project access. I have seen sites with twenty or more registered site administrators. I believe this occurs more as a symptom of long term neglect than an implementation issue.

I encourage clients to use the built in service accounts and APIs where practical.

You can plan ahead with all the requirements and the test lab set it up as a library, then go do multiple testing times, recording the default that's in the system. Later, go back to check the coverage you are missing, so you can plan ahead and maybe reuse the same set as next time. Sort of like creating templates and reusing them over and over.

We use the quality engineering testing tool plus the defect tracking to make our reports, projects, and quality better. Once we had the evidence to approve all the testing and all the coverage, the reporting went better. Usually, the products make it much easier to identify the issues we have.

It is nice, but it does have some weaknesses. It's a bit hard to go back and change the requirement tool after setup.

It's not flexible enough. The formatting is also an issue. For example, the project manager doesn't like the use it, even for requirements, because it's not easy for them to change it. If they make a mistake and go back, it is hard to change the formatting to make it good. So, they have to share or use another one that try to upload. But, after the upload, you cannot change it because the IDs are identified. It's hard for them to work somewhere in-between, adding something in there, then keep the rest of them record is still linked well.

It's difficult to change it. Let's say you set up the requirement, if you change the requirement, by adding any on bottom which won't cause an issue, but I want to add it in central somewhere, then you mess up all the linkage for the test plan and test lab.

This requirement piece is what I think is the biggest disadvantage for the Quality Center. I do know Micro Focus does have a bunch of the new tools, but that depends if a customer wants to change it, use a new tool or stay on an older tool.

Reporting is a bit complicated. They have a standard report, but if I don't want to use that, I have to use the Excel reporter.

I have used it for the start of the implementation at our organization using Quality Center versions: 8, 9, 10, and now, we're on 11.5.

We host it in-house, so basically we don't have any bad downtime. It runs mostly 24/7, so Quality Center is pretty good with stability.

So far, it hasn't been an issue.

I would give them a high score as they do a pretty good job.

In the Quality Center, there's a tool, which we started with, QuickTest Pro. From there, we started to use QuickTest Pro, later we introduced and evaluated it. It looked like the situation we needed.

However, we wanted tracking. We started with QuickTest Pro, but now we're doing this, which includes a lot of the different areas, like it handles the workflow and/or agile and involving many necessary departments.

I was involved in the initial setup. I installed configure, manager, and the patch providing user access, though now we have a team.

The setup is straightforward. It's not hard to set up. We even used the multi-complicated one because we didn't want have the database alone.

To someone looking at Quality Center, I would tell them: It's a good tool to use and the support is good. However, if you want a fancy and modernized tool with a fancy outlook, then Quality Center is not a good tool for you.

Most important criteria when choosing a vendor: personal style. I want to know who will be continually knowledgeable.

• They know what they are selling.
• They respond back quickly with accurate information.

If someone talks to me, and I try a few times, but I cannot get clear information from them, I may disqualify this vendor completely.

1> Intuitive GUI: Fairly easy to use and follow. For using QC, deep knowledge of the tool is not required. 2> For each test case, a test script with detailed steps can be created. This makes it easier to run the test script. 3> Provides interface with other test management systems like JIRA. 4> Excellent reporting process including customizable reports and charts. This is very useful for monitoring the progress of QA cycles and communicating the same to the higher management. 5> It stores test cases, test scripts, and requirements in a modular fashion, which can be easily copied and modified to create new test cases. 6> An extremely useful feature of QC is that it allows linking defects with higher level artifacts e.g., a defect can be associated both with a failed test script and the unmet high level requirement. It allows traceability of a defect with varying granularity of information. 7> No extra form required to perform searches on the defect list. Search is available for each field right on top of the list.

1> High licensing cost. 2> QC lacks a "watch" feature thus disallowing independent actors, such as managers / leads, to track the progress of issues. For example, for each defect, only the assigner and the assigned receive any updates / notifications. Everybody else has to employ external means e.g., e-mail to get these updates / notifications, thus introducing humans in the loop.

I have used HP Quality Center (QC) for over 3 years in an industrial setup. QC is a versatile Quality management tool that offers test case and defect management capabilities along with a customizable reporting process. It also integrates with other defect and requirement tracking tools, making it a good fit in multi-team environments, as well as integrated application environments. QC does have an involved initial setup, but once it's done, it is fairly easy to use by testing and defect management teams. However, as QC comes with a high license and maintenance cost, it is more suitable for large projects in terms of cost.

We have been involved in a lot of IT projects which need test management and for the test execution process, we are using Micro Focus ALM Quality Center.

We are able to use Micro Focus ALM Quality Center for test management, defect management, test process, test governance activities, and requirement management. We are able to achieve all of this, the solution is very useful.

Micro Focus ALM Quality Center could improve how the automation process works. Addiotnlally, the parallel execution needs to be optimized. For example, if multiple users, which are two or more users, are doing an execution, while we execute the cases, I have seen some issues in the progress.

Most enterprise solutions are moving into the cloud and this solution could work on its cloud compatibility. For example, if I have an Amazon or a Google cloud, I would like to know how would it best fit into their cloud environment.

I have been using Micro Focus ALM Quality Center for approximately eight years.

Micro Focus ALM Quality Center is a stable solution.

I have found Micro Focus ALM Quality Center to be scalable.

The technical support from Micro Focus ALM Quality Center is good, they are responsive.

I rate the technical support from Micro Focus ALM Quality Center a four out of five.

The initial setup is straightforward, it was not any more difficult than other setups.

The enterprise pricing and licensing are reasonable.

I would recommend Micro Focus ALM Quality Center to others.

I rate Micro Focus ALM Quality Center an eight out of ten.

On-premises