Have you ever experienced the frustration of having to rerun an enormous test suite because a few tests failed during the previous run? This can be a time consuming task, especially when test suites contain 100’s of tests!
TestNG, an annotation based testing framework for the Java platform has quite a few features that make it a compelling addition to any development team. One of the more interesting features is its support for rerunning failed tests.
Unlike JUnit, TestNG creates an XML file, dubbed testng-failures.xml, which defines each failed test; moreover, this XML file acts as a test suite which can be invoked at a later point in time (presumably when fixes are applied).
For example, via the command line, TestNG can be invoked like so:
c:>java org.testng.TestNG -sourcedir ./test/java ./test/conf/testng.xmlBy specifying the sourcedir option, TestNG will scan Java 1.4 source files for annotations via JavaDoc-like tags. The last parameter is a path pointing to a suite definition, which essentially defines which tests to run.
===============================================
corp-regression
Total tests run: 1205, Failures: 6, Skips: 0
===============================================If there were 6 failures in a JUnit run, we’d have to rerun the entire suite of 1205 tests to verify any associated fixes. With TestNG, however, a new suite file was generated which contains the information needed to rerun those 6 tests. Because the testng-failures.xml file defines a suite, this file becomes the last parameter when invoking the TestNG runner.
For example the command:
c:>java org.testng.TestNG -sourcedir ./test/java ./o/testng-failures.xmlwill only run 6 tests and produce the following output:
===============================================
Failed Tests suite
Total tests run: 6, Failures: 0, Skips: 0
===============================================JUnit is certainly the industry standard for developer testing in Java; however, TestNG offers a host of compelling features that make it a worthy addition for effective developer testing.
May 31st, 2006 at 5:15 pm
Hi: Have you evaluated JUnit4?. How does it compare with TestNG?. What I like to know is that 1) If someone with no JUnit written yet but starting from scratch, use JUnit or TestNG?.
2) If someone has JUnit written using JUnit 3.8, say, which would better provide backward compatibility?.
Thanks
August 31st, 2006 at 9:05 pm
For a JUnit4 vrs TestNG comparison, check out In pursuit of code quality: JUnit 4 vs. TestNG
TestNG can convert JUnit 3.8 tests into TestNG tests too.