Broadcom Agile Requirements Designer Reviews

We are trying to create models, then get the maximum number of test coverage by getting the minimum number of tests. That is what we are looking for: Getting the minimum amount of test cases which provide the maximum coverage. We want to test the minimum amount of cases. 

As of now, we do not have a tool which can tell us this. We might end up overtesting, but we have no way of knowing. For this purpose, we use other tools as well, to show how well it integrates and contributes to the continuous testing process.

We are trying to establish a process. As of now, we have not come to a point where we are only using ARD with other tools. We are trying to find better ways of how to use it. We have just done a type of PoC, where we have some model and created a number of test cases, which gives us an idea of the test cases to be executed afterward. We wanted to use it digitally, in practice, but what we see is data integration points. This does not give us the flexibility of ramming the test with multiple sets of data at one time. Therefore, we are just trying to look into the aspect where it gives us more flexibility in terms of integration and using it in our continuous process.

Optimization is something which I like in the tool. It gives me an idea of what are the minimum number of cases which I can execute in order to complete my testing.

I like the optimization, because it gives me an idea of the minimum number of test cases. It also gives me an idea of coverage.

The integrations with other tools is very important for our continuous testing process.

I was looking into some pre-release testing on the integrations with CA Application Test, and when I exposed the test directly to App Test, I found that the tests which we directly exposed have one set of data that we can directly expose and run in App Test. Once we run it, we come to know whether the test has passed or failed. Currently, when we are running a test in App Test, we are running it with TDM data, so it helps my tests run with multiple sets of data at once. I feel that it is a more efficient process if we are running a test with different sets of TDM data. However, if you take data directly from ARD, it is one test with one set of data, which is getting executed. Therefore, data flexibility is something which I would like to see, along with more integration with App Test.

Also, a template in App Test should be created in advance. This has proven to be time consuming. The process is not fully automated, because there is a lot of manual intervention is required. I would like an enhancement or something from the tool that I can use in a more automated way than a manual process. 

It is quite a stable product.

We are fine in terms of scalability.

The support team is good.

I have not previously used a test case design tool.

The company was just using ALM for our test management: creating manual test cases, designing and planning, requirements, and traceability. 

The initial setup was quite straightforward.

I would advise to use this product, because it is a good way to create models and establish model-based process testing. The optimization technique with integrations is seamless, and a very good tool.

It gives us an idea of creating the visual diagrams, which are quite easy to use. It is helpful in creating our business processes. The optimization technique helps in giving us the minimum number of test cases with maximum coverage.

We use it primarily to model our applications to automatically generate test cases.

It's great. It's preformed very well. It's very useful.

There are three major benefits. The first one is there is a time savings. It's much faster to create the model than it is to generate test cases. It's much faster to update the test cases in the model than it is to update them manually. 

The second benefit is really around organization. It helps you organize the different parts of your application and it really forces you to understand how they work well together. 

The third benefit is it's a visual representation of your requirements, which is very useful when communicating with a business partner or communicating with another person inside the company, to help them understand the application.

The most valuable feature for us is the automatic creation of the test cases, based on the models. And the impact analyzer that shows you where your change is impacting your test cases, and the ability to export that to our TFS Instance.

We're actually participating with CA to provide some enhancement requests on new features we'd like to see. I can't think of any specific ones off the top of my head, but I know they are evaluating, and they have already provided us features based on our feedback previously.

In terms of improvements, just licensing. That's a big area.

There are some opportunities to improve the stability. The application will often times, if you put your computer to sleep and you open it back up, not be responsive any longer. You'll have to kill it and open it back up. But the good thing is that I've talk with CA about that, and they're actually fixing the issue.

They're very receptive to fixing the product. It's a fairly new, young product.

I don't see any concerns with scalability. It's really a local install. I don't have any concerns there.

Tech support is one of their strongest features. The people that support the tool, know the tool. The product owner is here at the CA World conference. I've met him; I already knew him from interactions with support. So they're very, very good and responsive as it relates to support.

My CA representative and I were talking about a problem I was having and he described to me how this tool could help with it.

This was a problem that we were having independent of CA products. It was a problem that we were having in our testing space. This was a tool that we weren't actively using but helped to resolve that issue.

Everything but the licensing was straightforward. The licensing is extremely complex. It needs work. That's probably the biggest issue. I know they are redesigning the licensing model, but that was a royal pain, to be honest.

When looking for a vendor, there are a few key things. 

• The features of the tool - do we already have something that exists that has that capability today?
• Cost, obviously - what's the cost? 

Those are the main areas.

I rate it a nine out of 10 based purely on the experience with licensing. If that were resolved it would absolutely be a 10.

Regarding advice to a colleague looking at this type of solution, I would say buy it. Start using it. I would give them an overview of what it does. There are great YouTube videos out there on ARD. That's how I initially learned about it, outside of the conversation with my CA rep. So go look at the YouTube videos, but then get your hands dirty. Get in there and start working with it, because it's really powerful once you learn it.

It removes ambiguity in requirements, meaning that there are fewer demands for clarifications afterwards. We are using it to improve communication between business, tests and development.

For testing, it gives a very good view of the test cases and shows priorities of action so that the most important things are tested first. This helps us to save time as well.

The best example of improvement would be better communications available for my project.

When you have long flows or models, the display can be problematic, as it necessitates zooming in on the GUI and you lose the full display.

As your model grows in size, it becomes increasingly difficult to get a meaningfull view of the entire model. You might have a view with the whole model, but then the text and flow will be incomprehensible. Likewise, as you zoom in, you lose the entirity of the model. A workaround is zooming in as much as needed and scroll around from there, but one could wish for a better solution.

I have been using the solution for about 10 months.

We haven't encountered issues with stability but we have experienced some problems with our JIRA project, which takes too long to load because of its size.

Larger models do require significant time for analysis.

I would rate the technical support as top notch. They are very responsive and they help us solve problems quickly with brainstorming discussions which is amazing.

I didn't previously use another solution.

I was involved in the setup and there were some security issues which proved to be complications because we are running it with VMware and there are some difficulties with offline registration and license revalidation.

We have Test Data Manager but didn't evaluate other solutions.

You should have a detailed discussion with the vendor as to what your actual requirements are. Also, concerning test data management integrations, there are quite a few valuable features which require TDM so you should consider if you need the full value trail there.

Primarily, we use it to convert our test process to model-based. It has actually changed our entire process from script-based to model-based.

It has been pretty dramatic. We have rolled it out across many of our digital teams. In the first year, we saw about a 70% reduction in our script creation time. So, we went from five days per sprint to one day, one to one and a half days per sprint in the script creation. So, it has been really effective.

1. The co-monitorial modeling. It lets us reduce the amount of scripts that we need to execute to get complete coverage. That is really important to us. 
2. Traceability from requirements through test cases.

1. A better alignment to compliance-related requirements and being able to get visibility into those. It is really important for our industry. 
2. Test data: There is Test Data On-Demand and there is the vTDM solution. We would like to see a nice integration between the data and the models. That is really going to help us in terms of CI/CD.

Stability is good. Any issues that we have had in terms of rollout, the CA team has been there to support us all along the way. All the solutions solved our problems so that has helped a lot to ensure we are using the product right. Where there were gaps, they closed them pretty quickly. We had the CA team and the whole group combined spent a lot of time with us, and they have been really responsive. From that perspective, where there were issues, we have solved them pretty fast.

I am not seeing any real issues here. Early on, we had implemented in a way that was not very scalable. We quickly corrected the way we had it implemented. We would love to see it go to a SaaS solution. It is just easier from a procurement and onboarding perspective. However, so far, the scaling has been fine.

Make sure CA is with you regarding the support.

We are a large legacy company with a lot of legacy software. Converting to DevOps and agile, obviously, we had to get faster to the market. Our mind was already wrapped around how we enable teams. When we found the solution that fit nicely into where we are trying to go with our development process overall, it just kind of evolved. 

I was involved in the initial setup. What is good is there were some consulting services to make the complex a little more simple. We have some real talented guys on our team that were more instrumental in that rollout than I was. So, it was not that difficult. One of the more difficult things is just getting people to think in terms of drawing models instead of writing scripts. Changing that mindset was something that we had to do, but CA helped us through that as well.

There is the open source community, then there is the traditional HPE product suite that we are already using. However, there were not a lot of products that did model-based solutions like this. Specific to this tool, probably just HPE.

Most important criteria when selecting a vendor: 

The tool capability is important, but we also assess the risk of the company. For example:

• How stable are they? 
• Are they enterprise-wide? 
• Do they understand a company of our scale? 
• Can they scale with us? 

That is really important. 

The second thing I look for is support. It is really difficult to affect change if you are in it on your own trying to get it done. So, having a team there beside you which helps facilitate the change is really important. I think CA has done a good job with that.

One of the best features is that you can actually develop some of the business designs in the application and transfer that into scenarios if you don't go out of the detail level. And you can see the coverage. You can get your test cases automatically done for you.

I believe the best thing is that you can automatically generate your test cases.

Because of the way it is designed, all the boxes are displayed no matter how many there are. So, the more boxes you create the smaller and smaller they are displayed and you cannot even read it. So it's impossible to design your requirements. If you cannot read the boxes, what's the point?

Then, they need to make it more user-friendly, so that you can navigate from one section to another. Also, being able to automatically create stories. Some of the feedback I received from my team is that you cannot go out of the detail level. For example, if you want to create test cases from this level, it actually stays at a really high level unless you add a hundred steps. And then you have a huge process diagram that you have to maintain after that.

I haven't actually installed it, so I can’t give you much information about that. But, I've seen that its not able to do some things. For example, you cannot create user stories inside Agile Requirements Designer. We just saw another product where you could drill down to BBD and TDD. And you can actually transform that and create your user stories to be able to do that and this product is lacking that.

We haven't actually used technical support yet.

Well right now we use multiple systems. For example, if you want to build your business diagram, you have to use Visio. You can’t export the diagram into a format useable by Agile Requirements Designer. You need to go directly to an application lifecycle management system and build your steps, but you need follow the business diagram you created in Visio to be able to create those steps for those test cases.

So you have to upload all your requirements into an ALM system to be able to identify your metrics. But with this tool, I see the benefit that you can actually do it automatically. To be able to identify all scenarios, you need to test.

We just saw another product called Blueprint that is very similar to this. It was very user friendly and it has a lot of capabilities for where we're heading, which is agile. It’s very user friendly, the navigation is really good, and you can read the boxes.

Because of the amount of time that you have to invest to maintain it and because its not simple, not user friendly, I would rate CA Agile Requirements Designer a 5/10.

There are tons of really valuable features. First of all, it's a model-based testing tool and the market of model-based testing is pretty narrow, and there are few offerings. I did extensive research in this area and at the moment CA Agile Requirements Designer is probably one of the best tools in the market. It allows you to utilize the full power of model-based testing using a simple method to build a workflow, which is more familiar to most software and test engineers than to business analysts or product managers.

It has a native engine that allows you to configure test coverage for your application model, and you can analyze this coverage by applying different optimizations. This really helps to make informed decisions when you're choosing the right amount of testing you need to apply. How not to over-test your application from one point of view, and from other point of view, how to guarantee 100% functional coverage in your testing. This is really crucial. Another great feature is that it can be integrated with test automation tools. This tool is agnostic can be integrated with a number of different products.

Another useful feature is the ability to integrate with a test data manager. For example, in my array, I don't have large databases from which I can get enough data for my testing. But other companies do have large data sets and they can integrate their testing tools with the databases to do sub-setting and masking. CA can also build synthetic data for testing. These are very powerful features that you can't find in other products. In this way, CA is in a unique position, because they offer these unique products with unique feature sets.

We have a number of challenges and one of the methods we use to solve these is model-based testing. That's what we're researching right now. We're doing a proof of concept with Agile Requirements Designer, which is pretty feature rich. ARD was a startup and CA bought it a little bit more than a year ago. Since then, the quality has greatly improved.

I don't know what improvements to make since we are not yet using this in production. We are still in the preliminary stages of testing. I plan to do more research on it. They're already doing the right thing. They're improving the integration with automation and that’s really important. However, the documentation is not clear regarding best practices, so that could be improved. Since it’s not likely that every automation framework can be easily integrated with CA ARD, it would be nice to have recommendations regarding what changes to make to the automation framework in order to successfully achieve that.

I’ve used CA for approximately for two months.

I haven’t used it for a very long time. The first version I tried had a lot of bugs. The version I tried in June or July was much more stable.

I think that it is scalable.

We worked with several people from CA and they did a really great job helping us familiarize ourselves with the tool. At the conference, I spoke with maybe five or six representatives; technical guys, sales guys, even feature developers, people with strong technical backgrounds and I got a lot of new information. It's cool. They're doing awesome stuff.

We hadn't used model-based testing tools before.

I am not the person who approves the budgeting, but I am the person who'll do the research and present the results to the decision makers.

We didn’t evaluate any other alternatives for model-based testing.

I think that this product is going to be very good, because of the team who develops it. They are trying to identify the important features the market wants and to implement them. They are also researching the best approaches to apply to particular cases.

If you're looking for model-based testing, I would advice to first to take a look at CA ARD. It's not a very complicated solution, so your learning curve will not be too steep. You’ll be able to quickly identify the core functionality. First, limit yourself to just model-based testing, then you can dive deeper and research different integration capabilities, such as how to integrate with automation and data management.

Our primary use case for this tool is designing our manual test cases.

CA ARD is a shift-left tool for us, toward BA, business analysts, where we actually create our requirements. We use it for creating our test cases and QC. Once we get the requirements from a client, we design our flowcharts in CA ARD and then it automatically creates our test cases and QC, because there's an integration in between CA ARD and QC.

We used to provide RTMs to the technical team or the testing team, but now, CA ARD is an extra helping hand. We can provide the exact scope of testing or development to the technical team. That enhances things at the organizational level.

CA ARD has some beautiful features which I haven't found anywhere else. For example, when designing or creating our test cases and doing scenarios, we are able to restrict our flows. If we take a data link between two processes, we can actually restrict it, so that in production, if our functionality breaks down, we can restrict that, and all the flows related to it will be removed from the test data set. That is something we can actually do from CA ARD.

We are using many tools in our projects, like Tosca, but CA ARD is a very closed tool. It doesn't provide integration with Tosca. The possibility of creating a test case and exporting it into Tosca is not available. Integration with end-to-end automation tools, like Worksoft or Tosca, is not provided by CA ARD as of now.

Although there are many integrations which are already available, like Jenkins for automatic performance testing, where we can actually build batches and all, still, there's room for improvement here which the CA ARD team can work on.

In terms of designing, it is very stable, but we are still exploring automation. We just started using the automation feature of ARD.

The initial setup depends on the kind of scenario you are working on. But I found it straightforward. I took about one-and-a-half weeks to study the tool and I'm fine working on it.

Deployment for our clients takes around 30 minutes. It's relatively quick. It generally takes one to two people for deployment. Once the system is up and running it takes two or three people to maintain it.

We have four to five people using the solution. We have an automation team of around 11 people. About half the team is using this tool.

I have to become more familiar with the automation part of CA ARD, and then maybe I'll come to the point of saying that it is a good tool which enables you to take your automation, the testing and development, from deployment and designing of your test case, to pushing them into your testing module - whatever tool you are using; that it is an end-to-end solution.

I would rate this solution at six out of ten. I'm working on Tosca, which is also an end-to-end solution for testing, and I see Tosca as a better option to use, except that Tosca is very costly compared to CA ARD. There are other tools as well, like RPA for process automation. CA ARD is lagging in competing with these tools. As an automation tool, it might not be the strongest.

We use CA Agile Requirements Designer for model-based testing, and also automate test case creation and generate test automation scripts.

It reduced the effort required for test case creation, test case review, automation script generation, and maintenance of test cases and automation scripts. We were able to reduce our QA team size by 50 percent. It helped resources without programming knowledge to create automation scripts. It enabled in-sprint automation and implement continuous testing.

It helped us to move from manual testing to automation testing. Application level SMEs without scripting knowledge were able to create automation scripts. Integration with Test Data Manager, reduced the effort required in synthetic data generation and finding the test data from a master testbed quickly. Integration with Rally helped us to upload test cases quickly.

• Business flow creation
• Test case optimization and generation
• Risk-based testing
• Automation test script creation
• Test Data Manager integration

The product reduces the efforts required in documenting requirements, test case creation, and automation script generation. Test Case Review was quick and easy, by visually going over the flow. We were able to standardize the automation scripts.

ARD is constantly improving, and CA is always working on adding new features. Integration with Agile management tools can be improved, i.e., mainly test case maintenance and linking test cases to the automation script.

No stability issues.

No scalability issues.

Technical support is excellent. I have been using this product for nearly four years. They provide solutions quickly for issues encountered.

We sought out this solution because we needed to automate our regression test cases. Application SMEs without automation scripting experience were our main challenge.

When we got introduced to ARD, we did a PoC and realized the benefits. Then, we implemented ARD in our projects.

The initial setup was straightforward. Installation was quick. Then, we needed to activate the license by following the instruction provided.

This tool reduces the cost associated with test cases, automation script generation, and maintenance costs. So go for it. It will bring great savings to your organisation.

We did not evaluate any other solutions.

If you are looking to implement model-based testing, reduce the effort required for documenting the requirements, test case creation, automation script generation, maintenance of test cases, and automation scripts, then go for it. It will bring great savings to your organisation.