APM Tools Utilization in Performance Testing: PerfCast - Winter 2017

QASource QASource | December 27, 2017

“Monitoring is the first step that leads to control and eventually to improvement.”

- Anonymous

What’s New

  • HPE LoadRunner has released its new version 12.55 which is loaded with following new features:

    • Support for MQTT & CoAP protocols (M2M/IoT connectivity protocol)
    • Support for running JMeter scripts
    • Existing protocol enhancements

    Read More

  • SmartBear has released a new version 4.70 of LoadComplete with following new features:

    • Cloud testing enhancements
    • Support for mobile apps performance testing
    • Performance improvements in generating test results

    Read More

  • Neotys has released a new version 6.2 of NeoLoad with following new features:

    • Native GIT Support
    • Graph Substitution in NeoLoad Web Dashboard
    • Trends in NeoLoad Web

    Read More

 

Tool Evaluated By Team

Utilizing APM Tools in Performance Testing

Using APM tools in QA cycles can immensely help the performance testing teams for in-depth analysis of the application’s health before it goes to production.

In this newsletter, we have evaluated one of the major Application Performance Monitoring (APM) tool [New Relic] that monitors and manages the application’s performance. It also provides the trending data about the application's performance with detailed transactions information & reports which helps to troubleshoot the performance bottlenecks.

New Relic

Key Features

  • Provides support for Java, .NET, Node.js, PHP, Python and Ruby technologies
  • Support for Windows, Linux, Android and iOS platforms
  • Analyze the threads using thread profiler tool to identify potential bottlenecks
  • Offers variety of capacity, database, scalability and web transaction analysis reports
  • Major KPI (response time, throughput, error rate and apdex score) trends accessible on dashboard
  • In-depth information on heap/non-heap memory usage, GC, active/idle thread count and http sessions
  • Customizable alert policy for individual and group alert notifications
  • Alert system integration with applications like Slack, HipChat, Campfire and Pagerduty
  • Option to view and download reports on daily, weekly and monthly basis with detailed downtime information

Read More

 

Knowledge Base

APM Tool Selection Criteria

In today's competitive market, various tools are available with a variety of features which may or may not suit your application requirements. Below are few important parameters that can be considered while selecting an APM tool for your application:

Evaluation ParametersDetails
Application Support
Support for technology and hosting environment of the application
Transactions Analysis
Level of analysis/drill downing of threads for finding out the root cause of the problems
Reporting
Include detailed analytical components and ability to customize the reports
Automated Actions
Ability to take automated actions upon breach of threshold. For eg. Restarting the application instance upon consecutive 'OutOfMemory' exceptions
Third Party Applications Integration
Ability to integrate with third party applications like Slack, HipChat etc.
Alert Notifications
Ability to customize the alert messages and send in various ways like SMS, Email etc.
Role Based Access
User role based APM console access
Pricing
Flexible pricing and option to add/remove additional features
Update Process
Regular updates and seamless upgrading process
Customer Support
Responsiveness of customer support
 

APM Workflow in SQA

APM Workflow in SQA
 

Best Practices to use APM Tool during Performance Testing

Below are the best practices that can help the performance testing teams to efficiently utilize APM tool and uncover bottlenecks early:

Best Practices to use APM Tool during Performance Testing
  • Use APM tool in QA cycles for full stack monitoring of application components during load testing
  • Target critical scenarios from business perspective and monitor the performance
  • In-depth analysis of APM data for weak links or transactions to identify the root cause of the performance issues
  • Put the load on server from cloud to test geographical response time discrepancies
  • Test response time for all the third party applications embedded in the application
  • Add alert policy or rules to get an alarming information on the notification channels
  • Add major KPI trends on dashboard for quick access
Have Suggestions?

Have Suggestions?

We would love to hear your feedback, questions, comments and suggestions. This will help us to make us better and more useful next time.
Share your thoughts and ideas at knowledgecenter@qasource.com

New Call-to-action

Disclaimer

The logos used in this post are owned by the individual companies of each logo or trademark and QASource claims no rights to ownership of the logos. Nor is QASource sponsored by, or associated with the owners of the logo, and uses them for informational purposes.

This publication is for informational purposes only and nothing contained in it should be considered legal advice. We expressly disclaim any warranty or responsibility for damages arising out of this information and encourage you to consult with legal counsel regarding your specific needs. We do not undertake any duty to update previously posted materials.