Home | About Us | Stelligent  

TestEarly Weblog

July 2006


Developer Testing and /Owens26 Jul 2006 03:48 pm

I had a lot of positive and negative responses to my most recent blog entry comparing Ruby to Perl and Python (Newbie to Ruby?) In fact, I had one guy go to the next level by creating a forum about me and the validity surrounding my intelligence of programming…since I am just in marketing. It’s ironic that this same guy wrote an entry on his own blog about shampoo and conditioner; more specifically marketing Herbal Essence. Maybe I should create a forum questioning his knowledge of hair products (and masculinity :) )…since he is just a programmer. My overall point is that I think Ruby and Rails are great at what they do but ultimately you should chose whatever technology is the most comfortable for your core development team. That being said, I am again discussing Ruby…this time with a Rails focus and have elected to follow a more impartial approach.

I heard the phrase “Don’t Drink the Kool-Aid” the other day while engaging in a very non-technical conversation. The phrase seems appropriate when assessing the hype that Ruby on Rails has generated. It appears that everyone is drinking the RoR Kool-Aid. The growing interest is fueled in part by the simplicity and functionality of Rails. Having all the functionality built directly into the web application framework eliminates the need for verbose configuration files. Arguably the productivity rate gained from Rails is its best case. Programmers are productive when they’re happy and they’re happy when they’re productive. Drink up!

Not everyone is drinking the Kool-Aid…just not yet at least. Try convincing a Fortune 500 company to adopt Ruby on Rails. Their arguments will usually go something like this:

  • Infrastructure is lacking
  • Lack of tool support and resources
  • Lack of a good IDE
  • Productivity gains from dynamic languages are immaterial
  • Immaturity

The main argument usually surrounds the maturity of the framework, but like anything else these issues will almost certainly get resolved with time. Once it’s built up more in small and medium enterprises their defense will most likely be justifiable.

Ultimately, the best way to judge Rails is to experience it while building an application. Once Rails has the time to mature, it will be ready to be more of a contender in the enterprise space. I may have been a little bold in my original entry on Ruby describing Rails “in position to overtake Java.” Maybe I had too much Kool-Aid that day. I don’t think Rails will ever displace Java, but I do believe it will be a strong player in the future.

Developer Testing and News and /Glover25 Jul 2006 08:44 pm

Quality Labs has released a new version of NDbUnit. The 1.2 release addresses some enhancements and bug fixes. Full .NET 2.0 support is still being worked out; however, the framework appears to work without changes in most scenarios. See the forums for specific details.

This release had some excellent input from Steven Dehandtschutter, who joined the team as a committer, as well as contributions from Gerke Geurts and Martin Duncan. Thanks guys!

Developer Testing and Build Management and /Duvall25 Jul 2006 10:37 am

Many developers have experienced the “It Works on My Machine” problem when developing software. This is when you learn that some software you’ve been developing doesn’t work in a particular environment, but it works fine on your machine. For example, you may have tested the software on your machine, yet when the “same” software was tested on, say, the testing machine, it fails. There are many reasons this may occur. One reason may be that you forgot to commit a file to the version control repository. Another reason may be that the testing configuration is different - for instance, the application server’s pooling mechansim may be set for fewer connections. There can be many reasons, but in all cases, it means that something is different between your machine and the other machine(s). This is when you exclaim “But it works on my machine!” - as your last line of defense.

Introduce a similar, yet different, antipattern: “It only builds on this machine” or as I like to call it: The Magic Machine. The Magic Machine is the only machine capable of building your software. It’s the “Don’t touch that machine or we’ll go out of business” machine. It turns out this is much more common then we may realize.

This occurs for one reason: tightly-coupled build dependencies!

Typical Causes:

  1. Build script is tightly coupled with the IDE
  2. No build script. Every build is a custom hack job.
  3. The build script only exists on the Magic Machine
  4. A customization of the build script only exists on the Magic Machine
  5. Dependencies on Database/Application Server or other servers that must be recreated for each machine
  6. Custom configurations (username/password, scripts) on Magic Machine

The AntiPattern
As you see in the image below, there is a tight coupling with the various dependencies. This is all well and good unless you try build on a different machine with these same dependencies (which were hard-wired on the magic machine)
Magic Machine

Symptoms and Consequences

The symptoms are very straightforward: you try to build on a different machine than the magic machine and it won’t build. The bottom line is that if this Magic Machine ceases to work, you are effectively out of business for the product you are producing. The risk of relying on only one machine that can build your software occurs more frequently than you may imagine. You must immediately work to ensure that you can build your software on any machine. Depending on your particular circumstances, this may be a significant undertaking.

I highly encourage development groups to build on a separate machine; just make sure it’s not the only machine that is capable of building your software. The worst-case scenario, and all too common, is if your development machine acts as your build machine (and magic machine).

Refactored Solution

  1. Create one set of build scripts that everyone uses
  2. Build dependencies into the build script (e.g. using relative references)
  3. Make it simple to download servers and other software for which the build has dependencies
  4. Scorch the environment on each machine to ensure build is working and not relying on files from previous builds

Some will say, “Why don’t you create a backup or Ghost image?” and replicate the builds this way. While creating backups or images is a good practice in general, it is not a good way to solve this particular problem. It may be effective as a way of reducing some of the risk associated with a magic machine, but backups or images will not solve the underlined problem because of the constant changes in software development.

Benefits
The primary benefit is that you have working software at any point in time. What’s more is you will learn of integration problems before you deliver to the users. By building on a separate machine, and any machine, you enforce a single source code line obtained through your version control repository.

Developer Testing and /Glover23 Jul 2006 08:39 pm

With the release of JUnit 4, the fundamental structure of a JUnit test has changed dramatically. The previously rigid rules which stipulated that all test method names must start with test and that fixtures be implemented by overriding setUp and tearDown are gone in favor of annotations, much like TestNG or .NET’s NUnit. This structural change, however, does not necessarily affect extension frameworks like DbUnit, XMLUnit, or HttpUnit to name a few.

If a framework was designed with a delegation API, then things will work smoothly. For example, take DbUnit, which was eloquently designed with an API which facilitates creating database aware tests without having to subclass a specialized test case. Using the latest version of DbUnit in JUnit 4, which, by the way, was created long before JUnit 4, is as simple as defining a static method decorated with the @BeforeClass or @Before annotation which handles fixture logic (such as inserting data into a database).

Remembering that DbUnit requires two things to work correctly (a database connection and a seed file), one can define a working JUnit 4 test case which utilizes DbUnit as follows:

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 final ApplicationContext context =
  new ClassPathXmlApplicationContext("spring-config.xml");
 dao = (WordDAO) context.getBean("wordDAO");
 handleSetUpOperation();
}

This fixture is run just once and does three things. First, Spring is initialized, second, a reference to a DAO is obtained from Spring and finally, DbUnit is invoked via the handleSetUpOperation method which obtains a connection to a database and inserts the desired data via the connection as follows:

private static void handleSetUpOperation() throws Exception{
 final IDatabaseConnection conn = getConnection();
 final IDataSet data = getDataSet();
 try{
  DatabaseOperation.CLEAN_INSERT.execute(conn, data);
 }finally{
  conn.close();
 }
}

private static IDataSet getDataSet() throws IOException,
 DataSetException {
  return new FlatXmlDataSet(new File("test/conf/words-seed.xml"));
}

private static IDatabaseConnection getConnection()
 throws ClassNotFoundException, SQLException {
  Class.forName("org.gjt.mm.mysql.Driver");
  return new DatabaseConnection(DriverManager.
    getConnection("jdbc:mysql://localhost/words",
      "words", "words"));
}

Tests can now easily be defined with the @Test annotation.

@Test
public void findWord() throws Exception{
 IWord wrd = dao.findWord("pugnacious");
 Set defs = wrd.getDefinitions();
 assertEquals("size should be one", 1, defs.size());
}

@Test
public void createWord() {
 IWord word = new Word();
 word.setSpelling("pabulum");
 word.setPartOfSpeech(PartOfSpeechEnum.NOUN.getPartOfSpeech());
 Set definitions = this.getDefinitionsForWord(word);
 word.setDefinitions(definitions);
 try{
  dao.createWord(word);
 }catch(CreateException e){
  fail("CreateException thrown while trying to create a word");
 }
}  

This class doesn’t extend DbUnit’s specialized test case either. One thing to note, however: in order to use a coarsely grained fixture via the @BeforeClass annotation, you must declare the fixture method as static, which means using static class members if they are to be used in concert with fixture logic, such as my dao instance from above. This is in stark contrast to TestNG which is more flexible and doesn’t demand the static declaration.

If a JUnit extension framework predates the 4.0 release, it doesn’t necessarily mean the framework doesn’t work with 4.0. In fact, any framework should work and those that provide an API are in excellent shape to provide maximum flexibility.

For more information on JUnit 4 see IBM’s developerWork’s An early look at JUnit 4.

Build Management and /Duvall20 Jul 2006 11:20 pm

I’ve seen a lot of build scripts over the years. I am surprised when good software developers that follow good practices when developing the application software seem to forget these practices when writing their build scripts. Without following some basic effective practices, these build scripts can get unwieldy in short order. If your build scripts are not maintainable, you may get diverted from developing the application software - at the most inopportune times. In no particular order, I’ve listed some of the more common bad build scripting practices I’ve seen. The examples are in Ant, but apply to any build scripting platform. In this first entry, I will demonstrate five “bad build practices”. My next entry will include the other five bad practices I’ve observed…and, of course, I have committed from time to time :)

Bad Practice One: Hard-coded values
This is a definite no-no. For instance, in hard coding a value, you could make the script dependent on a particular directory being present on each computer. Without changing the script, you’d need to copy these files to each computer on which you build. To eliminate the use of hard-coded values, use relative references and be sure the files/directories are checked in your version control repository so that they are available from your local workstation.
Bad:
<property name="config.dir" value="C:\projects\finanit\implementation\config" />
Good:
<property name="config.dir" value="${basedir}/config" />

Bad Practice Two: Copy and Paste build scripts for different environments
For instance, you may need different build behavior for each environments - such as development, QA and Staging. Some choose to copy and paste their build script and make the minor modifications in each script. In doing this, you’ll experience the same problems when copy and paste programming. One solution is to put the common behavior in a single build script and use a .properties file to adapt to each environment. The concept is similar to using a base class for common behavior in OO programming. The left side of the image below is the “bad” in which the same copy of the file has been used and slightly altered for each environment. In the right side of the image below, the build.xml has been generalized to account for different environments. You make the changes to the .properties files instead of different copies of the build scripts.

Copy and Paste Build Scripts

Bad Practice Three: Long targets
Just like long methods decrease the maintainability and understandibility of your source code, so do long build targets in your build scripts. Try to break up build behavior into discrete targets.

Bad Practice Four: Not cleaning up after yourself
If you are not removing temporary files, classes, assemblies and other files before you build, you have no way of ensuring the build was successful. If you need to get the same version of a build at a later time, you may have difficulty in creating it. What’s more is that you may not discover a defect (or chasing a latent bug) until later in the development cycle.
Good:

  <target name="clean">
    <delete dir="${classes.dir}" quiet="true" />
    <delete dir="${reports.checkstyle.dir}" quiet="true" />
    <delete dir="${checkstyle.classes.dir}" quiet="true" />
  </target>

Bad Practice Five: Building with the IDE only
There’s no problem with using your IDE to build the software. In fact, I’d encourage to use it to build. However, if your build script executed from your IDE is different than the build script you run from the command line (e.g. using Ant/NAnt) then you are asking for problems. This is particularly difficult to manage if each developer has his own variation of the build script(s). Building software should be consistent and repeatable. An effective build process begins with all developers using the same build script(s).

Build Management and Continuous Integration and /Duvall11 Jul 2006 08:29 pm

Steve Matyas of 5AM Solutions and I developed an Ambient Orb Ant task that changes the color of an Ambient Orb based on a build event. There are many uses for an Orb, but the purpose of this task is to change the color, animation or comment of your Orb when a build passes or fails. If you’re creative, you can configure your build script to gradually change the color of the Orb based on certain conditions (e.g. you are exceeding your code duplication percentage threshold). The Ambient Orb Ant task is hosted at Quality Labs.

Ambient Orb

So, rather than just a binary representation (such as X10), there are virtually limitless combinations of colors and animations with an Ambient Orb.

The task is setup as a listener that listens for build events (pass or fail). Consequently, when you define the task, you are registering the task to listen to build events. Based on the build status, the Orb color is changed. Because it checks the build status, you probably want to execute this task as one of the first tasks in your build script - in case your build fails early. For instance, you may choose to use your init task in Ant.

Steps to using the Ambient Orb Ant task
1) Download the Ambient Orb Ant task
2) Place the extracted jar in your Ant class path
3) Define an Ant target and configure as demonstrated below:

<target name="registerOrb">
  <path id="orb.class.path">
    <fileset dir="${ant.home}">
      <include name="ambientorb-0.5.jar"/>
    </fileset>
  </path>
  <taskdef classname="org.qualitylabs.ambientorb.ant.OrbTask"
     name="orb" classpathref="orb.class.path"/>
  <orb listener="org.qualitylabs.ambientorb.ant.OrbListener"
    deviceId="AAA-99A-AAA"
    colorPass="green"
    colorFail="red"
    animationFail="heartbeat"
    animationPass="none"
    commentPass="The+build+passed"
    commentFail="Build+Failure!!" />
</target>

In this example, if the build passes, your Orb’s color will change to green (12) or red (0) if it fails. Pretty cool, huh? In the example, you’ll need to change the deviceId attribute. This attribute is the unique identifier located on the bottom of your Orb. Notice the use of the plus (+) sign to indicate spaces in the commentPass and commentFail attributes. Go here for additional documentation. A short movie demonstrates the installation, configuration and use of this task.

The project is open source and free to download and use. If you’ve got ideas for improving it, comment here or feel free to join the Quality Labs project.

News and Continuous Integration and /Glover and /Duvall and Podcast03 Jul 2006 03:39 pm

The Stelligent Early Quality Podcast Series kicks off by asking Disco Paul Duvall, who’s the lead author on the upcoming book “Continuous Integration: Improving Software Quality And Reducing Risk”, what is CI?

It’s amazing how many copasetic phrases from the 70’s can be worked into a 5 minute show!