Why use active and passive monitoring for a web site?

Question

When using a RUM solution (such as DynaTrace, AppDynamics, NewRelic, etc.), what are the benefits to add an active solution with probes/robots?

Randall Hinds · Accepted Answer

Some good info in comments here; sharing from my personal experience to augment.  Having used three of the Dynatrace APM product (DCRUM, Synthetics & AppMon w UEM coming quick), I can say there are distinct use cases for each.

Not to be terribly repetitive, DCRUM's passive approach has served my team well but is reactive in nature.  Packet capture and analysis of live traffic can't detect much proactively.  Dynatrace has replicated many measures DCRUM provides in AppMon & UEM, and our source at the vendor reports far less investment in DCRUM development.  This is sad because DCRUM is perhaps the most affordable product offering they have, and it can be configured to the capacity of HW Infra of the licensed instances.  We use it for every app's traffic flow, as a result.
Synthetics (formerly Gomez) is an active approach but can produce frequent false positives.  ISP back bone and/or Agent Node issues are common enough to cause some companies to instrument two or more competing synthetic/robotic measurement products for isolating those painful false-positives.
AppMon is their deep-dive agent which measures at thread/method level calls from within the app engine (JVM or .NET).  Once instrumented, the agent is a great tool for triage and performance monitoring.
UEM is a code injection approach that is layered (& licensed) on top of AppMon to allow click/error/response measurement at the client browser.  This is probably the most difficult of their APM product to instrument correctly, but it can become an engine & data source for all audiences at a company to leverage.

To your specific question - Combining active & passive measures gives you a chance to compare real user transactions to robotic transactions and rule out false-positives (ISP backbone issues should not be counted against application availability, and likewise a Node Agent not running a script should not impact your 99.999% up time goal).  
Robotic measurements are generally set at wider intervals than DCRUM polls, too.  Comparing measures could give an App Support team evidence to show an impact only lasted 2 min (smallest DCRUM polling interval) as opposed to 15 min (avg script interval).  When shooting for five 9's of availability, every minute counts. :-)

** This is based on my personal experience and does NOT represent or imply the view of my employer **

it_user181782 · Answer

Actually you should use RUM to have a end user real experience and robots are used to capture data when you do not have traffic in your website or if you want to stress it.

it_user69183 · Answer

Great answers here. The only thing I'll add is that probes or robots allow testing before deployment, something that shops of your size frequently ignore as being too difficult/costly/technical for the existing staff. They also step outside the 'self-diagnosis' problem. Logically there is a limit to self diagnosis capabilities. (Hint: how do you know you're not drunk? This is a classic problem as the very mechanisms for self-analysis are impaired and unreliable. A sober friend or breathalyzer is needed.)

Daniel Newkirk · Answer

Hello,

The primary benefit of deploying robots is that they allow you to test that the application is up and available.  They can also test that key application functions are working.  Robots also allow you to test application availability during off hours, after application recycles, etc.  This allows you to detect and resolve application problems BEFORE prime time, when the majority of your users are accessing your system.  Active monitoring can provide other benefits as well, but this is the most critical.

All of the major vendors can capture every transaction with their end-to-end / real user monitoring solutions.  When it comes to transaction profiling (measuring each transaction hop-to-hop as the transaction traverses the application the back end), Dynatrace can capture every transaction.  This feature is all the rage right now.

This feature also has its limitations.  In my 30+ years of experience monitoring applications across all kinds of platforms and technologies, capturing too much performance data can be as much of a problem as capturing too little data.  Why, because 98+ percent of the data never gets looked at.  Also, many application performance issues get written off as "that's the way this application works" and/or, "it's too expensive to fix", etc.  Once people start ignoring performance data and performance alerts, they tend to ignore everything.  It's just human nature.

What you really want / need to know on the back end is when transaction hops exceed normal / expected performance.  To accomplish this, you need to profile what normal / expected performance is.  When performance exceeds normal performance thresholds (and in some cases way under performs), you need to be able to quickly alert the appropriate people / teams and have them diagnose the root cause.  Sampling, can effectively accomplish this task.

This is not intended as a criticism of Dynatrace, they have a legitimate monitoring approach.  My point is only that the other vendors also have legitimate, time proven monitoring approaches as well.  You need to decide what is important to you and your company.