As headlined in The New York Times only a couple of weeks ago, the Iowa Caucus debacle was “a systemwide disaster…” and the “app used to tabulate votes may have been inadequately tested.” While additional facts are still coming out, the entire nation watched as there were major delays obtaining and reporting on the Iowa results. And then there was uncertainty as to whether or not they were accurate. While the intent of creating this app was supposed to be positive – to enable simpler more streamlined reporting, faster – we know that’s not at all what happened. And, that’s not exactly how you want your new mobile app rollout and brand to be remembered.
Could this have been prevented? If there was adequate testing of the mobile app before launch – it most certainly could have been prevented and there are lessons to be learned here for anyone developing or enhancing mobile apps.
Over the last few years, mobile application testing has become an essential “must have” versus a “nice to have.” With so many variables today such as a growing myriad of devices, different versions of devices, evolving bandwidth, etc. there’s a growing concern among app developers is to make sure appropriate, thorough and documented testing is completed – before launch.
According to one NY Times article, “It wasn’t so much that the new app that the Iowa Democratic Party had planned to use to report its caucus results didn’t work. It was that people were struggling to even log in or download it in the first place...” Perhaps launching a new app to 1,700 people with different levels of tech savviness with no testing or training was a big risk. And, that was not the only risk. There were apparent concerns by cybersecurity experts that the app had not been properly vetted by independent experts or tested at appropriate, real-time scale before being launched in Iowa.
So how could this have been prevented?
Here are 7 testing recommendations that, based on what we know, would have helped prevent this incident:
- It’s critical to test not only your new mobile apps but any enhancements on different network bandwidths 2G, 3G, 4G, etc. so you know that the app will work across different bandwidths. Many Iowa Caucus users had issues downloading and signing into the app which could have been due to performance issues on slower bandwidths.
- Test the load, stress and concurrent usage of the app. This is likely why there were problems tabulating and sharing results from roughly 1700 sites.
- Perform thorough Usability Testing. It sounds like there was not enough real-life testing and training on using the app before the caucuses so when it came to “crunch time", there was frustration and chaos.
- Obtain Platform Certification with specific security patches. They should have had a list of smartphones and security patches required on the phone for the app on which the testing was done. There is no indication that this was obtained prior to launch.
- Test for memory leakage and storage issues. There were some references that the app got stuck on the very last step when reporting results, which was uploading a picture of the precinct's results. This could have possibly been tied to the specific phone that user was using as well.
- Perform Volume Testing. There were reports that while the app was supposedly recording data accurately, it was reporting out only partial data which could have been due to the sheer volume of the data being collated all at once.
- Complete thorough functional testing based on the requirements of caucus math. The app apparently showed different numbers than what they had submitted as captured in their screenshot, which would have been caught with appropriate functional testing.
What’s the Moral of the Story?
Ask yourself if it’s worth the risk launching your mobile app without thorough QA testing. Whether you’re an established software development company or a startup without an established QA function, consider partnering with a company like QASource to make sure your mobile app goes through all of the appropriate testing protocols to identify bugs or any issues in early stage development. It’s critical for your users and for your brand to ensure everything works in real-world environments.
QASource helps Fortune 500 companies as well as scores of startups launch better quality software faster. Schedule a quick call with one of our experts to LEARN MORE.
Check out our report 5 Genius Ways Product Companies are Using a QA Partner to
learn how product companies use a QA partner to launch better software faster.
Download your free report below!
