Understanding how lines are (un)covered
While it is common for projects to measure what percentage of their statements are executed by tests, this single number is woefully inadequate at providing a detailed understanding of the extent to which a project’s code is tested, if there are gaps in the tests, and if these tests are useful in any meaningful way. For instance, seemingly simple changes in one part of the codebase may reduce the efficacy of existing tests that seem otherwise unrelated.
Code coverage can be useful to track long-term trends in how tested a project is, but on a day-to-day basis, can’t serve as an indicator for the change in test suite quality. In particular, moving the coverage needle even 0.01% can be extremely difficult in a project with millions of lines of code. At such a large scale, focus often drifts from which lines of code are covered to simply the number of lines covered. However, a change in the coverage of several hundred critical lines of code might be important