Code Coverage and /Brothers23 Apr 2008 02:43 pm
Interesting Clover Behavior - wiping out coverage database between test tasks
This was discovered with Clover 2.2.1
We have a build.xml file with the following pseudo-flow:
- clover-setup
- // this causes all future compilations to use the clover compiler to instrument the files.
- compile
- javac _all_source_
- test_A
- javac _unit_test_source_ // yes, this is a subset of all source
- junit _unit_tests_
- test_B
- javac _unit_test_source_ // yes, this is a subset of all source, and redundant with test_A
- junit _integration_tests_
- clover-report
- NO DATA!
By inserting < clover-log / > at the end of test_A and test_B, I figured out what was going on.
- after test_A, we have a nice coverage level
- but at the end of test_B, we have been reset back to 0
My hypothesis is that there is a bug in Clover that causes it to:
a) wipe out all of the coverage data thus far when new code is compiled.
b) prevent new code from updating the coverage database.
And sure enough, by removing the javac compilation step from test_B, we have an accurate combined coverage report.
Moral of the story:
- Do all of your compiling before you run any of your automated tests
- < clover-log / > is your friend.
April 23rd, 2008 at 3:52 pm
[…] Test Early wrote an interesting post today on Interesting Clover Behavior - wiping out coverage database between test tasksHere’s a quick excerpt This was discovered with Clover 2.2.1 We have a build.xml file with the following pseudo-flow: clover-setup // this causes all future compilations to use the clover compiler to instrument the files. compile javac _all_source_ test_A javac _unit_test_source_ // yes, this is a subset of all source junit _unit_tests_ test_B javac _unit_test_source_ // yes, this is a subset of all source, and redundant with test_A junit _integration_tests_ clover-report NO DATA! By inserting […]
April 23rd, 2008 at 4:48 pm
[…] Search Engine Land: Must Read News About Search Marketing & Search Engines wrote an interesting post today on Interesting Clover Behavior - wiping out coverage database betweenHere’s a quick excerpt…a) wipe out all of the coverage data thus far when new code is compiled. b) prevent new code from updating the coverage database. […]
April 23rd, 2008 at 4:50 pm
[…] unknown wrote an interesting post today on Interesting Clover Behavior - wiping out coverage database between …Here’s a quick excerptThis was discovered with Clover 2.2.1 We have a build.xml file with the following pseudo-flow: clover-setup // this causes all future compilations to use the clover compiler to instrument the files. compile javac _all_source_ test_A … […]
April 23rd, 2008 at 5:55 pm
[…] Search Engine Land: Must Read News About Search Marketing & Search Engines wrote an interesting post today on Interesting Clover Behavior - wiping out coverage database between…Here’s a quick excerpt…a) wipe out all of the coverage data thus far when new code is compiled. b) prevent new code from updating the coverage database. […]
July 3rd, 2008 at 6:38 am
Hi.
I’m a developer at Atlassian and work on Clover. This is not a bug in Clover but a feature!
By default, coverage is only measured since the last compile. If you wish to change this behaviour, you need to set your span setting. More info on this here: http://confluence.atlassian.com/display/CLOVER/Using+Spans
The reason for this behaviour is that the flow of development activities is normally one of compilation followed by funning junit tests and this sequence is then repeated many times. If, by default, we were to count the coverage preceding the most recent compilation we would be displaying coverage that was no longer accurate.
Your setup is different and needs the tailing of the span setting as above.
Hope this helps!
Michael.