Transaction trace under APM.
Ajax section under browser
Transaction trace under APM.
Ajax section under browser
Helped us pin point the exact piece causing the performance bottleneck by using the transaction trace view.
Instead of picking up a few values under trace, it can provide a list of every hit and their traces while keeping minimal overhead.
6 Months
Yes. Installation of plugins seem to be really difficult due to the documentation available.
No issues.
No issues yet.
Yes, installation of the APM piece and Server piece is straightforward but the plugin may cause an issue.
Trends and Alerting have been valuable features.
We can analyse events after the event has occurred. This gives us better visibility to plan resolutions in a timely fashion.
Performance/bugs.
6 months.
Sometimes we've noticed agents fail to report/respond.
Yes, we have encountered issues with stability.
No issues with scalability.
Excellent.
Technical Support:Excellent.
No.
No, simple setup process.
In-House.
Unsure.
< $1000/month.
None.
We can place ad hoc queries to understand user behavior and customer patterns. The data can be collected for any activity that we want to see on any part of our website.
Its greatest benefit comes from its ability to provide trending/marketing data. We can replace some requests from our data warehouse and get a quick glimpse into that data, especially with our mobile platform and website for different browsers.
The metrics that I want are already pushed to New Relic, but some data we haven’t pushed to New Relic. Internally, we have a solution that’s better than Insights to provide us analysis of that data. I’d like to see Insights take care of that analysis and use its graphic engine.
Not enough working experience to comment on this, but in my experience, no instability.
I haven't had to use it.
I wasn't involved in the setup.
The things that I would need are proprietary, and I don’t think that data can be pushed out to Insights. Take a look at internal processes to see what can be done quicker by Insight and make a good test case of it. You must present it as a better alternative to the existing issues. Get early adopters excited about it and build a relationship based on that.
For the way we use it, we like the monitoring of websites and it allows us to look at the trace stacks to identify problems. We’re also able to compare the day and the week before to see whether problems are reoccurring.
Before New Relic, dev and ops were separate, and now they’ve come together more and there’s less finger-pointing. There’s a clear understanding of where the problems are and so the people responsible can solve the issues. There’s more trust between the two groups and people are more willing to work together.
Stack traces don’t go far enough. They get to a point, indicate a question mark, and then stop. But New Relic is working on it. Also, one can get lost in the data.
We’ve been using it for two to two and a half year.
We've had one or two occasions of a few wide spaces in the graphs where the response time didn’t keep up. Sometime it’s a bit slow in processing data, but now it’s much faster and more stable with new releases. Some of the releases I think are already there in production.
We’ve been able to scale without issue. We’ve installed it and it works.
I haven’t had to use technical support. When we decided to use it, our CTO went to FutureStack and got a lot of knowledge.
So easy, very straightforward. Our developers, at the beginning, created some templates for data they wanted to track and that was it.
Just go for it. When I show people what it can do, it’s mind boggling. What’s coming up in the new release -- wow!
The most valuable features for us are:
When an app goes down, we can get insights into the issue with New Relic. It tells us what the problem is. For example, if there is an issue in the code, we see a spike in the error rate in the applications. The load environment lets us stress test our application to find the bottlenecks in the code.
They already have everything we need, so I can't suggest an improvement.
I was not involved with deployment.
Very stable. No problems.
No problems.
One issue – there is no option for live chat or phone support. We can go to the community forum to post a question and wait for the answer.
The solution was already in production when I got to the company.
New Relic is the right solution in my opinion. The light version of server monitoring is free with New Relic, and application performance index and error rate are the key features to look into.
I find the error monitoring of IIS web applications to be extremely useful. Being able to filter errors by URL, server, and period of time has been extremely helpful in quickly isolating and fixing problems. Being able to see a list of slow transactions is also very helpful in identifying the root cause of application performance problems.
In one case, a developer had an end-user report a recurring issue with a web application after a new release. I was able to use New Relic to find the error and provide the developer with the exact line of code that was causing the error within minutes of the issues being reported.
I can't say as I haven't used all of the features.
I've used it for one year.
No issues encountered.
No issues encountered.
No issues encountered.
I haven't had to contact customer service.
Technical Support:I haven't had to contact technical support.
We didn't have a solution in place prior to deploying New Relic.
It was straightforward.
We didn't install it through a vendor.
New Relic shows how much time each SQL transaction (SQL query) is taking to execute. By this, client can pinpoint the query and modify so that it takes less time to execute.
New Relic has detailed analysis of various parameters such as Web Server, Database server, Application Server etc. so we were able to provide the client with more productive reports.
Additional functionalities and application up time.
No major issues.
Yes, sometimes.
Yes, sometimes.
Used rarely, was average.
Technical Support:Used rarely, was average.
Cost effectiveness.
Straight-forward, though not completely.
In-house.
Good.
Check for scalability and application up time.
Over the years I’ve come to rely on information radiators during testing to get immediate (or as quick as possible) feedback from the systems I’m testing.
Firebug, log files, event logs and many other sources of information are all very useful to a tester. They can give you insights in to what is happening in the system under test.
We’ve just taken this a step further by rolling out NewRelic on our test servers.
NewRelic is what’s termed a “Application Management Solution”.
I’ve been talking about this internally as a system that can give us three distinct insights:
I’ve probably over simplified the tool and doing an injustice but it allows me to clearly explain the value we’re seeing from it.
User Experience Information
NewRelic gives us all sorts of data around how the experience is for end users when they use our product.
We can use this to ascertain how our product is being experienced by our customers, but we can also use it to understand how the experience is stacking up for our testers.
If we are testing and we observe a slow down we can check whether it really was a product slow down using NewRelic and more importantly; what’s actually happening on the stack.
We can use NewRelic to work out what browsers are being used across all of our environments. We can see the test coverage we have across browsers and we can also see what browsers our own business use from our pre-production test environments (where we test all kits before live deploy).
We can also then see which browsers are faster than others. We can see which versions are used and which browser is our most heavily used. Interesting stuff to help guide and tune our testing.
Server Information
NewRelic monitors the actual servers giving all sorts of information such as memory, CPU, process usage etc etc. This is great information on our test servers, especially during perceived slow downs or during a load test.
We have other mechanisms for measuring this also so this is the least used function in NewRelic when testing.
Product Performance Information
For me, this is the greatest information tools like NewRelic offer; they show you what the product is actually doing.
It includes what pages are being dished, how fast are they being dished, where they may be slow (in the DOM? Network?), what queries are being run, what part of the code is running them and how often they are being called.
When we dig around in the data we can find traces that NewRelic stores which give an amazing level of detail about what the product is/was doing when the trace was run.
It’s going to become a testers best friend.
In a nutshell what it allows us to do is provide an accurate picture of what the product is doing when we are testing. This means we can now log supremely accurate defect reports including traces and metrics about the product at the moment any bugs were foud.
The programmers can also dig straight in to any errors and be given the exact code that is generating the error.
We can see which queries are running meaning that if we encounter an error, a slow down or something worth digging in to we have the details to hand.
It’s still early days using the tool but already we’ve had deep insight in to how the product runs in our environments which I’ve never been able to get from just one place.
It’s immediate also. Test – check NewRelic – move on.
Imagine how powerful this could be on your live systems too.
Imagine the richness of information you could retrieve and imagine how fast you could get to the root cause of any problems. It’s powerful stuff. Expect to hear further posts on how tools like this can inform tests, provide a depth of supporting information and provide help to performance testing.
Some notes: