What needs improvement with Apica Synthetic?

Apica is costly, and there's no way to test mobile applications through Apica. You can test mobile applications through other platforms. However, Apica is only for APIs and applications.

We use ZebraTester to build scripts. ZebraTester is a bit complicated compared to LoadRunner’s VuGen. Initially, parameterization and correlation activity are somewhat difficult to do with a new tool. Once we get used to it, it's easy to build the scripts. The tool does not provide automatic correlation features. However, LoadRunner does. It finds out the dynamic values in the script and helps us to do the auto-correlation. It would be helpful if Apica could provide such features.

I have noticed that the tool isn't widely recognized outside our organization. Also, there aren't any tutorials or dedicated resources for this tool, making it challenging for newcomers to learn. It would be beneficial if someone experienced with it could provide guidance. Without proper support, it's hard to utilize its capabilities fully. I've found it's not widely used beyond our organization. Despite my experience with similar tools, I couldn't find any mention of it elsewhere when exploring opportunities for my project. I believe it has the potential to be a widely used tool, especially when compared to other performance-testing solutions like LoadRunner and JMeter. Here, the results are more precise, and the execution process is straightforward.

They need to come up with a single challenge initially, like a human ID, which becomes challenging. Then, we have to rest switches to a very low priority check. Some URLs, like HR functions, require a proper human ID for basic operations such as login and accessing data. However, regular functionality is sometimes restricted based on different organizations. Therefore, they should develop a functional ID management system that encompasses all types of checks, regardless of whether it's functional or non-functional.

It's not very user-friendly. There are more areas of improvement as well. We could use more detailed information in the request and response sections. Also, with correlation, there are times we can't correlate values in the header, and that would be a helpful improvement.

Apica should add more features and integrations with different tools and certain ticketing systems, like ServiceNow. As a user, you would want to add checks in more locations.

When we started using Apica Synthetic, we lacked a very important feature that was readily available in the following months. The alerting is usually very good - it allows if any websites or web pages are not responding properly. What we missed was the ability to aggregate alerts. This means that if we were monitoring one website with multiple probes worldwide, like Tokyo, France, and London, then Apica Synthetic would initially alert us if any of these three probes were responding with errors. We wanted a less sensitive alerting. For example, we could be alerted if the website was unavailable from all three probes, not just one. However, an aggregated check was not initially available. We do have two main open topics with them, regarding the features that we would like improved or added. * The reporting part that we use for our executives needs a bit more customization capabilities. Right now, you can use only the three main templates for reporting. We would like to be able to customize them. * The management of their single-sign on authentication does not 100 percent fit our requirements.

The first thing that I would suggest they improve is the user interface. Not from the point of view of how to access the features, but how they are presented. The user interface is very clear, but there are some components of the user interface that are not up to date. Just to give you an idea, today we have web applications that are called single-page applications that are much faster than the old style of web application. If we can move faster into the flow of the graphic user interface, and in a more effective way, it will save us a lot of time. Another feature they can improve is related to how easy it is to set up what they call on-premises locations. Apica offers locations all over the world, but they also offer a manual to install a location on-premises to check the availability of services that are not public. This process, at the moment, is not so easy to achieve. The last time we did it, we were forced to contact their support to set it up. The automation of this kind of setup is not good. It should be something that does not require human involvement to follow the deployment. The possibility of being totally independent in installing and using an on-premises location would be much better.

We have had some use cases come up, like when we have teams logging on through a VDI or multi-factor authentication where we have to think about things a bit differently. We are still working through how we might leverage Apica for those types of use cases. However, generally speaking, it has enough flexibility to be able to monitor the complex apps that we typically use it for. We have been focused on reducing polling times for synthetic checks. We have gone from 10 minutes down to five minutes for a pretty broad swath, but there is some appetite to reduce that further, which could be an improvement.

We have been able to work with the different protocols. Some of them have been challenging to use inside of our network. Sometimes, we face some difficulties getting to a particular site or authentication. The scripting features are proprietary. I would like to see more enhancements where the core solution can take on more generally available open languages, like Python. While Apica does accept Java and some Python, I would like to see more of the ability to just execute external scripts for generally available languages out there in industry. Learning the tool has always been a little difficult from a scripting perspective because the framework is proprietary and unique. Once we became used to what it does and how to perform it, then it became easier for my team and me. I would like to see some of the testing steps be part of a more well-known language, like Java or Python. That would be a big improvement.

The accuracy of alerts can be improved a little bit. Right now, it's good in terms of alerting immediately about failures or changes in response times. However, what we have seen happen is the number of alerts that we are getting is very frequent, and we would like to tone down the number of alerts. An option to aggregate alerts for a check from multiple locations is not available and creates duplicates. As a platform, it does send us good alerts, but it could be improved a bit. For the Synthetic platform, their dashboard could be a little better. I don't have many options to configure my dashboard to make it suitable for a power user versus a leadership person. The dashboard for our CTO is the same as the one that our technical staff is using. We have made this request to Apica, and they're going to make some changes to it. However, at this point, the dashboards and the way they present the checks could be better. For the level of metrics that it provides for each check, they could provide it at a step level or page level on the landing page rather than having us click a few more times to get that data. That would save a lot of time. These two would be really good changes to help increase efficiency from the current tool and current features that it provides for synthetic monitoring. As far as I am aware, the Apica platform doesn't allow us to execute scripts from JMeter or LoadRunner out of the box, but they do offer conversion utilities. For the LoadTest platform, they could increase the efficiency in terms of the results produced. The reporting structure of the results could be improved a bit. Apart from that, it is a pretty good platform.

Alerting needs improvement. It's a little noisy. It needs some better options. Currently, they have an issue, when you set up a Synthetic monitor, you can set up where it's monitoring from a data center that Apica owns. However, for each data center that you attach to a monitor, that's considered an extra license. That's a bit iffy. They're usually behind on the version of Chrome that they're using for the Synthetic monitors. Currently, they're using Chrome 85, they're 11 iterations out of date. They're trying to get that fixed up with something called Evergreen, which will basically be a Chrome browser that'll stay constantly up to date, but it hasn't been implemented yet. The problem with that is that we generally test our product with the newest versions of Chrome and everything like that, so sometimes we've run into issues. Also, when they updated to Chrome 74, we lost some monitoring capabilities that we had before that did not transfer over with this new version of Chrome. I'd like easier access to the API. Their API, it's not bad, it's just bulky. It's a little unwieldy in the way it has to be used. One of our app developers is currently working with them and he wanted to do a number of calls to the API, and he was not able to do that. They had to make special changes to our API to make the number of calls he wanted to make. It didn't seem to be scaling as well as we thought it would. But they worked with us to actually get it to do that. That's a plus point. I'd like to see more abilities to do mass changes to checks in the GUI, in the interface. Things like setting a mass amount of blocks for checking a bunch of checks and saying, "Make sure that this URL is blocked on all these checks." Currently, we can only do that through the API, and last time we had to do that, we actually had to use Apica support to do it. Finally, they have an audit log system called Journal. However, it can only check, if I remember right, two weeks at a time. That becomes really difficult when you need to check on something that you need to go back multiple times and you don't know the exact dates of the thing that changed. For example, I had a user who got changed in one of my checks and I needed to find out when it got changed. It ended up being three months ago, but I had to go back in two-week increments until I could find it. Their Journal, their auditing system, needs a little bit of work.

There is room for improvement with the GUI. It's not a big deal, but it would be great to fix the way the GUI is loading. Sometimes when we want the manager alerts and manager checks, it takes time to load all the way. With the whole GUI, if the information appeared quicker, loaded faster than it does now, it would be great. Also, when it comes to the way the internal agent is installed, because you can install an application on a server, I would love to see the application Docker-ized. If you could install internal agents using Docker or using containers, it would be easier for us to manage them and spin up internal agents. Most of the applications we have now tend to be Docker-ized applications. I'd love to see Apica going that way with its internal agents.

Having to install an application on your desktop to utilize something like ZebraTester is a little cumbersome. It would be nice to see that become a web-based application. Having the documentation a little more accessible, and easier to digest by people who are just learning how to use the framework, especially when it comes to more complex or more edge-based cases would be really helpful to have. That is really it, but I think the other thing that would be really nice to have, and it's not necessarily a big downside is when the browser agents need to be upgraded, it would be nice if that just happened automatically and it was transparent and seamless to us and to our infrastructure.

The initial screen on their dashboard could have a bit more data, but this is a small thing. It could have more data, so we do not need to drill down to a screen behind that initial information. I would like them to get a little better on the user interfaces that we need to go into.