Is your application developing unknown bugs with each sprint for unknown reasons? Not sure if your product is market-ready?
Implementing code coverage by means of unit tests have long been the answer to the above questions. This helped to avoid unknown bugs, but to an extent.
With the inception of agile methodologies, teams started working in small sprints. Automated testing became popular and unit tests were eventually pushed to the back seat. Today, we have tools that perform code coverage from automated functional tests, regressions suites, exploratory testing and even unit testing.
These tools provide a good overview of a continuously evolving code through tables and graphs in dashboards, hence providing a commendable release confidence. The tools perform both unit and automated functional testing coverage on app’s code. Comparison from both verticals is now possible which lets a tester analyze the production release.
A Second Thought On The Traditional Ways Of Code Coverage
Unit tests were initially considered as the only metric for code coverage, which often led to poorly designed tests. Such tests are generally written with the intent of meeting just the system requirements. Unit testing is important, but only when quality goals are limited.
Code coverage that includes test automation as well as functional testing in addition to unit tests is a better alternate here. It ensures that the source-code meets the system requirements as both functional and unit tests are used to produce the code coverage. Test quality is bound to refine as coverage is more comprehensive than just based on unit tests.
New Trends In Code Coverage
As mentioned above, unit tests are limited when aiming at quality through code coverage. So, one must consider integration tests, acceptance tests and even manual tests as a criteria. This is now possible through SeaLights and even JaCoCo now offers the code coverage through functional tests.
SeaLights: This is a continuous testing platform. It integrates with the test environment and measures test coverage across all types of tests. SeaLights provides insights on the extent of the tests and the risk involved with untested code changes.
Features
- Code Coverage through automated functional tests and manual tests
- Test GAP Analytics
- Release Quality Analytics
- Quality Trends, Charts, and Dashboards to analyze results
- Integrates with Jenkins and Maven
Benefits
- Clear stats depicting what application code has been covered through QA automation
- QA can focus on automation areas which are untouched
- QA can see the result trends for existing code coverage
- Helps QA team to calculate functional test coverage
- Learn more
JaCoCo: JaCoCo is a code coverage tool that calculates coverage based on unit tests. It’s developed in Java language and provides support for Groovy as well. It provides good reports of code coverage metrices and calculates the percentage of code covered based on methods/branches/lines. JaCoCo adds the lines of code for coverage calculation in byte code of the Java class. When executed, it instruments the class files used in the test with the coverage information. This produces a jacoco.exec file which is then used to generate the code coverage report.
Features
- An open source tool
- Written in Java and works on covering methods, branches, and lines
- Can be executed and integrated with Ant, Maven, Jenkins, and TeamCity
- Provides report for last execution of tests
- Learn more
| Benefits To QA Teams |
|---|
|
Adopting code coverage tools has empowered us in decision making in following terms Reports generated from these tools inform developers about the code (branches/lines/conditions/methods) or inputs that are mostly, or never used by test automation scenarios. These reports help the teams prioritize the key areas. This ensures:
|
|
The QA teams are able to create new test cases of uncovered areas of code |
|
Dead code detection helps us in the following:
|
Tips To Implement Code Coverage
- Begin with a goal; don't strive for 100% coverage initially
- Aim for a tool that considers your functional tests for calculating coverage
- Focus on increasing code coverage with each release cycle
- Use reports from previous analysis to make estimations each sprint
- Select a code coverage tool that supports your
- App’s programming languages
- Tools like Maven, Jenkins, bug tracking and project management tools
Have Suggestions?
We would love to hear your feedback, questions, comments and suggestions. This will help us to make us better and more useful next time.
Share your thoughts and ideas at knowledgecenter@qasource.com
