In a day to day work we typically run our tests and presumably check that we are getting the expected results.Our tests may all pass with flying colours, but if we've only tested 50% of the code, how much confidence can we have in it? So a step ahead would be to ensure how much of the application code is exercised by our tests which ensures the quality of the tests that we typically run and thus code coverage comes into picture.
What is Code Coverage?
Code Coverage is a measurement of how many lines/blocks of the application code are executed while the automated/unit tests are running.
Though code coverage is a white box testing methodology(as it requires knowledge of and access to the code itself rather than simply using the interface provided),
we might come across situations where we need to analyse/collect code coverage metrics for our functional test cases.This post briefs the need of code coverage,coverage analysis,instrumentation for functional test cases with an example using Cobertura.
How is Code Coverage achieved?
Code coverage is collected by using a specialized tool to instrument the binaries to add tracing calls and run a full set of automated/unit tests against the instrumented product. A good tool will give you not only the percentage of the code that is executed, but also will allow you to drill into the data and see exactly which lines of code were executed during particular test.
There are many coverage tools in market for Java Code Coverage -Cobertura,Clover,Emma,Jcoverage.In this post we will generate coverage metrics using cobertura -one of
the most widely used java coverage tool as it has edge over other tools.
- Open source Java tool.
- Can be executed from simple ant build script.
- Instruments Java bytecode after it has been compiled.
- Can generate reports in HTML/XML by class name, percent of lines covered, percent of branches covered, etc.
- Shows the percentage of lines and branches covered for each class, each package, and for the overall project.
What is instrumentation?
Instrumentation is all about manipulating the application code by injecting reporting code into strategic positions. In fact, the art of instrumentation falls in either of the two: class instrumentation and source instrumentation. Not surprisingly, the difference is that class instrumentation injects the reporting code directly into compiled .class files while source instrumentation creates an intermediary version of the sources which are then compiled into the final, source-instrumented .class files. Most of the code coverage tools will follow either of these instrumentation techniques.
For this post we are using a sample war file from (https://tomcat.apache.org/tomcat-6.0-doc/appdev/sample/)for instrumentation wherein for real time scenario,this should be replaced with your application war file.
Sample Cobertura War Instrumentation and Coverage in a web application
PreRequisite:- Tomcat 7 and above
- JDK 1.7 and above
- Ant 1.8 and above
- Cobertura 1.9 and above
- A sample war file for instrumentation/use your application war file
Pre-coverage steps:
- Extract classes from your war.
- Create an ant script for instrumenting the classes.
- Add required libs/dependencies for instrumentation (Sample attached- Refer build.xml and build.properties).
- Perform Cobertura instrumentation using your build target as below.
- Now we have instrumented classes generated as below ,Replace the generated files in the war (replace with the existing classes) and cobertura.ser file generated.
- Put your updated war in the tomcat webapps.
- Put your cobertura.ser file in the tomcat bin .
- Add cobertura.jar in the tomcat lib .
- Start your tomcat.
- Hit the required test cases (run ur web application flows either manually or through your automation scripts) (for this sample war -we are hitting http://localhost:8080/sample home page -this application has Hello page-hello class alone so we are executing a test to hit the hello page)
- Now put the updated cobertura.ser in the project ant home folder
- Run coverage targets as below.
- Now we have coverage for the test cases that we have run!
