I’ve written about three test categories (unit, component and system) for developer testing on a number of occasions- and I’ve even tried to draw a distinction between system and functional tests. Briefly, a system test verifies a software application from system end points, like web pages or web services- but they mimic the user or the end point protocol.

For example, you could write a system test for a web application with jWebUnit, which essentially mimics a browser by sending HTTP requests. Here is a jWebUnit test, written in TestNG, which logs into a web application:

package test.com.acme.web.cvg.ng;

import net.sourceforge.jwebunit.WebTester;

public class LoginTest  {
 private WebTester tester;

 /**
  * @testng.before-class = "true"
  */
 protected void init() throws Exception {
  this.tester = new WebTester();
  this.tester.getTestContext().
    setBaseUrl("http://acme.nfs.com:8080/cvg/");
 }

 /**
  * @testng.test groups = "system"
  */
 public void verifyLogIn() {
  this.tester.beginAt("/");
  this.tester.setFormElement("j_username", "lucy");
  this.tester.setFormElement("j_password", "lou");
  this.tester.submit();
  this.tester.assertTextPresent("Logged in as Lucy");
 }
}

This tests logs into an application and asserts that specialized text is present. While this may seem like a functional test, it isn’t because doesn’t act like a user- no browser is involved. This is as close as it gets, however, to a browser and accordingly, these tests are extremely valuable.

A functional test, as opposed to a system test, doesn’t mimic the end user- it is the end user. In essence, a functional test drives the application under test via a browser or even drives a GUI.

Also too, the functional test/system test line is usually divided by who can implement them. Almost always, system tests are somewhat clear box oriented- they are usually written in a development language and thus assume a fair amount of programming skills. Functional tests, on the other hand, are usually more friendly to author.

Selenium, for instance, is a functional testing tool, which provides a simple platform for verifying application functionality. Tests are authored in tabular format, much like that of the Fit framework. In fact, I think Selenium’s table format is easier to understand. Much like the xUnit paradigm, tests are organized into suites- through these suites, a test execution order can also be established.

For example, the following Selenium test is equivalent to the jWebUnit test from above.

The first row specifies a URL to open and the next two rows indicate form values to be filled out (j_username set to “lucy” and j_password set to “lou”). Next, a command is issued to hit the login button and then web page text (”Logged in as Lucy”) is verified.

Suites then can be created, which, in essence, map to use cases. For example, imagine a use case for user account creation. First, successfully login to the application and verify that worked. Next, create a new user and verify the user was created. Lastly, clean things up (i.e. delete the user so the test can be run again).

A Selenium suite simply points to tests- in this case, I have a suite that points to three logical tests:

Click on the image for a larger view

Selenium’s intuitive tabular format for declaring tests makes authoring functional tests a breeze. For more information, see Selenium’s documentation and an excellent article on IBM developerWorks.