Developers today have access to a plethora of tools for measuring code quality through static analysis, code metrics, code coverage, and many more. These tools make it easier to achieve a higher code quality — assuming that you know how to use them. Though these tools may be useful if utilized correctly, coverage reports can also be highly misleading, making it vital to be able to understand the reports.
What is code coverage?
Code coverage is measured by determining the number of lines that are run through unit tests. This is called line coverage, which is the measurement most tools focus on. Line coverage simply reports the percentage of lines of code executed by taking the number of executed lines over the total number of lines of code. For example, if a particular method is 10 lines long and the unit test ran 7 of the lines, then we would say the method has a line coverage of 70%. This works the same with classes as well: let’s say a class has 500 lines and only 200 of those lines were executed, the class would have 40% line coverage.
Branch coverage is another type of measurement that coverage tools report. This measures the coverage of decision points and conditional logic, such as blocks containing logical ANDs or ORs. It measures which decision outcomes have been tested. Branch coverage is important because it ensures that no branches lead to any other fault or abnormality of the program’s functionality and it validates that all branches in the code are reached. Branch coverage is calculated by taking the number of decision outcomes executed over the total number of decision outcomes.
When analyzing coverage reports, one important thing to keep in mind is that high coverage means there was a high amount of code executed; it does not necessarily mean a high amount of code executed well. Developers should understand that it is critical for them to not only learn how to use these coverage tools, but also how to analyze the analytics they provide.
Benefits of coverage reports
While analyzing code coverage reports, the following are some of the benefits a team can reap:
- Can be used to figure out ways to improve the code or tests for the next phases of development
- Helps a development team set goals (e.g. getting to a certain amount of coverage, or amount of unit tests that need to be written) and then monitor progress toward those goals
- Allows your team to evaluate and get a general idea of where potential problem areas or bugs are in your code
It is important to remember that code coverage reports do not necessarily indicate the quality of the code. However, these reports can generate useful facts on where to focus during the next phases of development. From a testing perspective, the reports provide a blueprint of the codebase and allow you to examine exactly which parts were neglected by unit tests. This allows developers to focus their attention on the untested, potentially most error-prone, parts of the code.
The point here is to try to get the maximum amount of code coverage on your entire codebase so you can see what parts of code have been executed. This will also let you examine where to add to more unit tests in order to cover more code.
What is JaCoCo?
JaCoCo is an open source toolkit for measuring code coverage in a code base and reporting it through visual reports. It is quite popular among the variety of code coverage frameworks out there. It measures line and branch coverage based on the code covered by running unit test cases and provides a visual report, including highlighted lines of code and the total percentage of code executed in each method. From this analysis the developer can assess the code that has not been adequately tested. It’s best to focus on the areas where the coverage is particularly low and try to analyze why this code hasn’t been tested fully.
Working with JaCoCo
JaCoCo uses the standard JVM Tool Interface. During a build a JaCoCo agent attaches itself to a JVM. When the JVM starts. and whenever a class is loaded, JaCoCo can use the agent to see when the class is called and what lines are executed. This is how code coverage statistics are collected. When the JVM terminates it creates the coverage report file. The reports are published in the directory /target/site/jacoco.
The report types that are supported are the following:
- HTML – Visual report for end users
- XML – designed for further processing by other external tools
- CSV – Basic report type suitable for creating simple graphs
One of the most useful features is the visual manner it presents its statistics in. JaCoCo can be integrated in various IDEs such as Eclipse or Android Studio by installing a simple plugin.
The report will also highlight the lines of instructions that have not been called and/or have low code coverage. This is illustrated by the screenshot below. The general color coding scheme that JaCoCo follows is the following:
- Green – fully covered lines
- Yellow – party covered lines (some instructions or branches missed)
- Red – lines that have not been executed at all
JaCoCo and other coverage measurement tools are great to have in a developer’s tool belt. JaCoCo saves the developer’s time by pinpointing the exact location where the code coverage is low, providing a more specific area to focus on. The JaCoCo tests do not take long at all to generate as they are created while your unit tests are running. By taking a few minutes to install the JaCoCo plugin you are really going the extra mile to help make your code better covered and tested. This tool should be instrumental for helping you progress in the development process in order to create more high quality software.