Cloud Infrastructures for Automation Testing

Quarterly Automation Testing Expert Series - Vol 3/4 2021

Timothy Joseph
Timothy Joseph | September 29, 2021

With an instant drift towards remote working culture, many organizations face challenges in utilizing their existing automation infrastructure for cross-platform testing. The major concern is with the mobile apps that run on a variety of devices, hardware, OS versions, and screen sizes.

Before the remote working era, QA members used to either share devices for manual testing or run the automation suite on multiple devices using a hub. However, these practices are difficult to implement when working remotely as engineers are working from different locations and everyone needs specific devices to complete a comprehensive testing strategy.

Automation Testing Challenges When Working Remotely

Automation Testing Challenges When Working Remotely
  • Validating different components of an application on multiple devices simultaneously
  • Delays in releases or incomplete testing due to time constraints
  • Increased costs to provide the same devices to multiple engineers
  • Risk of infrastructure or device handling and shipping
  • VPN issues
  • Increased chances of cyberattacks due to insecure Wi-Fi

Cloud Testing Market Trends

Gartner predicts the following worldwide trends based on cloud end-user spending:

Worldwide Public Cloud Services End-User Spending Forecast

20000 40000 60000 80000 100000 120000 140000 160000 180000 0 46131 50165 53121 46335 59451 71525 102798 122633 145377 14323 16029 18006 59225 82023 106800 50165 53121 1220 2046 2667 15.15% 54.36% 41.42% 25.71% 80.33% 118.61% G r owth in 2 years Desk t op as a Se r vice (Daas) Cloud System Inf r astructure Se r vices (Iaas) Cloud Management and Security Services Cloud Application Se r vices (Saas) Cloud Application Inf r astructure Se r vice ( P aas) Cloud Business P r ocess Services (B P aas) Millions of U.S. Dollars 2022 2021 2020

From the above trend, we can see that the DAAS (desktop as a service) and IAAS (infrastructure as a service) will grow tremendously by 2022 as most of organizations will shift their infrastructures to cloud. This is primarily development, DevOps, and testing. Below are a few benefits that teams can expect when shifting towards a cloud-based development and QA strategy:

  • Less setup cost and maintenance
  • Quick setup and access
  • Easy sharing between engineers
  • Heightened security
  • Continuous access
  • Easy to upgrade
  • Less prone to loss of data in case of natural calamities

Cloud Testing Infrastructures for Automation Testing


  • BrowserStack

    Developers and testers use BrowserStack to perform cross-browser, as well as cross-platform testing. It can be integrated with multiple automation tools and provides features such as parallel testing, CI/CD integration, and test reporting. While testing mobile apps, we can test geolocation, push notifications, network simulation, and gestures-based testing.

  • AWS Device Farm

    This testing infrastructure created by Amazon is used to test web and mobile applications. On this platform, teams can run the automation scripts created using different automation tools. AWS provides features like parallel testing, CI/CD integrations, and test reporting. AWS device farm provides a smart feature to test android mobile apps which automatically crawls every page of an app, takes a screenshot, analyzes each view, and shares the results.

  • Firebase Test Lab

    Firebase test lab is majorly centered on testing Android and iOS apps. It provides advanced features like parallel testing and CI/CD integrations. Firebase test lab has an interesting feature called Robo test, an automatic crawler that verifies the structure of the app's user interface. The tool captures logs, screenshots and prepares videos to debug the issues.

  • Xamarin Test Cloud

    Developed by Microsoft, the Xamarin test cloud is a testing infrastructure that concentrates on native and hybrid mobile apps testing. You can use it with different mobile automation tools. It provides features like screenshots, logging, CI/CD integration, and parallel execution. Xamarin Test Cloud also supports Azure active directory integration for easy organization.

Best Practices to Use the Cloud-based Testing Infrastructure

  • Use a private cloud to add privacy to your automation code and application.
  • Run the tests in parallel to make the most of cloud infrastructure.
  • Avoid the usage of static/hard-coded paths in the automation frameworks.
  • Avoid the usage of XPaths to speed up the test execution.
  • During mobile app testing, make sure to use the 'full reset' option before starting the automation suite.
  • Use a CI/CD tool with cloud-based infrastructures in order to preserve the data in case of switching to another infrastructure.
  • Utilize the APIs exposed by the cloud platform to fetch the analysis results like logs, performance data, memory usage, CPU usage while test execution and add the results in the framework's standard HTML report.
  • Avoid keeping login credentials or confidential data in the automation framework. Make sure to pass it securely at run time using CI/CD plugins like the credentials binding plugin for Jenkins.
  • Do not use cloud-based testing infrastructures while developing and debugging test scripts. Charges and fees are calculated based on usage time.
Have Suggestions?

Have Suggestions?

We would love to hear your feedback, questions, comments and suggestions. This will help us to make us better and more useful next time.
Share your thoughts and ideas at


This publication is for informational purposes only, and nothing contained in it should be considered legal advice. We expressly disclaim any warranty or responsibility for damages arising out of this information and encourage you to consult with legal counsel regarding your specific needs. We do not undertake any duty to update previously posted materials.