A client recently acquired a company with an embedded software product that, according to all customer comments and due diligence reports, was of high quality. It was the plan of our client to integrate the acquired product with its larger enterprise application. Several months into the project, an esoteric measurement of software quality raised its head: Change Tolerance.

Change Tolerance describes the ability of software to survive change. Rather than an objective metric, it is a subjective measurement that is rooted in a complicated combination of code complexity, code dependency, test coverage, build processes and automated testing frameworks.

Other composite measurements, such as Defect Resistance and Quality Risk also require specific knowledge regarding not only the code that is currently compiling, but also the path and processes used to bring that code to fruition.

Rarely do the history, processes and measured code quality result in an acquirer walking away from a deal…too many other factors influence a decision to acquire. But objective and knowledgeable assessment of the true quality of existing software provides excellent negotiating leverage.

We worked with our client to eliminate the issues that affected the Change Tolerance of the product. But those costs could have – and should have – been passed onto the seller, rather than absorbed by the acquirer.

A unique offering we provide to organizations who are pursuing acquisitions is a historic quality assessment; where we assess code and process quality both in present terms and as a trend over time in the organization. These assessments can show the impact of the loss of a “star” performer, the drop in quality due to aggressive release cycles, negative impacts from change of technology or – for the best companies – a demonstrated pattern of consistent and well-applied quality practices.

Other subjective measures such as accounting records, corporate values, quality of management team, competition and intellectual property can affect valuation. But, in the case of software technology acquisitions, software quality is unique in its ability to be supported by meaningful metrics and its direct correlation to the value of the acquired assets.

Engaging in a software acquisition without an outside, objective assessment of core product quality can leave millions on the table in terms of unaddressed valuation concerns.

My last two blog entries, Effectively Exploring Exploratory Testing and Reckless Test Automation covered two Agile techniques and the controversies surrounding their acceptance in the industry. I am continuing my discussion of practices championed by Agile approaches by further exploring Pair Programming (PP), the extreme of Extreme Programming (XP).

Pair programming can best be described as two programmers working together at a single workstation to develop software. Normally pairs switch roles and partners several times a day to reduce redundant work and the negative effects of incompatible pairings. While XP vehemently advocates this “shared-knowledge” practice, most who have not tried it are skeptical of its productivity claiming it to be an inefficient use of programming resources. Are there situations in which PP just isn’t as cost-effective? In essence, its argued that two programmers are doing the work of one. Can a pair really outperform two individuals?

Research into pair programming shows that pairing produces better code in about the same time as programmers working singly. Working on complex tasks, with two brains (and two pairs of eyes) is one obvious benefit of PP. In addition, pairing is a cost-efficient way to allow junior programmers to witness the skills of their senior partner through line of sight learning. There is also an overwhelming sense of peer pressure so partners are disciplined to apply themselves that much more. When you have someone sitting next to you it’s far easier to stay on track…just as long as “staying on track” doesn’t interfere with the occasional Test Early blog break.

Are two heads always better than one? Remote software development, along with workplace flextime and telecommuting are becoming increasingly popular. Although it is possible to pair program remotely its best if the core work be done in the same location. Of course, getting agreement on the coding standards may result in conflict for the pair. Most importantly though, the style of PP just does not suit everyone’s personality. It really does depend on the “chemistry” of the pair. While some people may prefer having another programmer constantly sitting over their shoulder other programmers absolutely loath this idea. Traditionally programming has been taught and practiced as a solitary activity. Imagine sticking an introverted programmer into this noisy, extreme, day in and day out environment. For limited periods, programmers can adapt to this setting but over long periods of time the process may seem exhausting to them.

In the end it’s all about acclimatizing your development process to what works best for the people that you have. When instituted in the right setting and with the right pair, Pair Programming will be advantageous. Demanding pairing to developers who refuse to change their coding habits just won’t work. In certain circumstances, some things just aren’t meant to be shared.

I’ll be giving a presentation on developer testing best practices in September at the SD Best Practices Conference and Expo hosted this year in Boston.

This presentation covers validating XML with XMLUnit, testing database dependent code with DbUnit, low level performance testing with JUnitPerf, and functional web testing with JWebUnit. We’ll also look at combining these various tools to build effective testing frameworks.

NAntAddin is a free tool which enables one to manipulate NAnt targets and tasks directly within MS Visual Studio.

Exploratory testing (ET), or testing when learning, design and execution are concurrent, is not the random and ineffective approach as once believed. ET is still a new concept and hasn’t been wholly embraced by the industry, but surprisingly it is one of the most widely used approaches in software testing. The problem is that very few people take it seriously and its real value is often misunderstood.

Exploratory testing has a highly situational structure as opposed to a planned set of requirements that is often favored. There are lots of myths (well, I will refer to them as myths) out there justifying that scripted testing (ST) and only scripted testing is the ideal and that exploratory testing is becoming obsolete. I am a firm believer that ET integrated with your existing approaches fits in nicely with the other tests testers perform. When we are faced with software with unknown requirements or software in an unstable condition, exploratory testing is a cost-effective approach and may uncover those vulnerabilities that traditional, pre-planned tests tend to miss.

As companies begin to seek more agile methods of developing software, the acceptance of exploratory testing will begin to increase. Until the acceptance rate of the value is better recognized questions such as when it should be applied and how to choose between ET and ST still need to be warranted. Unlike a scripted testing approach, exploratory testing follows an intuitive plan of observing, evaluating, and teaching. Testers usually have a general test plan in their mind as well as a fairly good understanding of the application and the business domain. ET emphasives flexibility and creativity as testers adapt to interacting test tasks.

It is important to realize that exploratory testing is not against the idea of scripting; in fact the most effective test strategies implement a hybrid ET/ST approach. There are many factors to consider when determining which approach to implement. Factors such as how well the tester understands the operation and complexity of a system, the skill sets of the team, and the degree of acceptable risk are important to assess when deciding which approach will be most conducive.

With the appropriate blend of both ET and ST the chances of successfully spotting most of the risks will be improved. The power of how effective employing exploratory testing will be is really based on the knowledge, experience and intuition of the individual tester and the variability among the test team as a whole.

NDbUnit is a .NET library, written in C#, which facilitates programmatically placing a database into a known state. This framework can be used to increase repeatability within developer tests that interact with a database by ensuring that the database’s state is consistent across the execution of tests.

Taking advantage of NDbUnit in VB.NET is a breeze, especially when combined with NUnit. For example, to use NDbUnit, all that’s required is:

• an XML seed file
• a corresponding schema file
• a database connection string

Using those three pieces of data, one can initialize NDbUnit via NUnit’s fixture mechanism:

<SetUp()> Public Sub init()
 Me.fixture = New OleDbUnitTest(connection)
 Me.fixture.ReadXmlSchema(schemaPath)
 Me.fixture.ReadXml(xmlFilePath)
End Sub

Now tests can be written that depend on the data found in the XML seed file. For instance, the XML seed file defines the word pugnacious:

<word>
  <WORD_ID>2</WORD_ID>
  <SPELLING>pugnacious</SPELLING>
  <PART_OF_SPEECH>adj</PART_OF_SPEECH>
</word>

<definition>
  <DEFINITION_ID>1</DEFINITION_ID>
  <WORD_ID>2</WORD_ID>
  <DEFINITION>Combative in nature, belligerent.</DEFINITION>
</definition>

<synonym>
  <SYNONYM_ID>1</SYNONYM_ID>
  <WORD_ID>2</WORD_ID>
  <SPELLING>belligerent</SPELLING>
</synonym>

To verify NDbUnit’s functionality, the following test case executes a CleanInsert operation (i.e. the data in the XML file is inserted into the database after each targeted table is cleared). After the CleanInsert, a different database connection is established and pugnacious is retrieved from the database and verified.

<Test()> Public Sub verifyWordInTable()
  Me.fixture.PerformDbOperation(DbOperationFlag.CleanInsert)
  Dim stmt As String =
      "select spelling from word where word.word_id = 2"

  Dim adapter As New OleDbDataAdapter(stmt, connection)
  Dim dset As New DataSet
  adapter.Fill(dset, "word")

  Dim table As DataTable = dset.Tables("word")
  Dim trow As DataRow

  For Each trow In table.Rows
    Assert.AreEqual("pugnacious", trow(0),
	"word spelling is not pugnacious")
  Next
End Sub

Note, NDbUnit isn’t a testing framework- it’s a database seeding utility to be utilized in concert with one. By allowing the state of the database to be controlled via NDbUnit, test cases can become repeatable, meaning they can run without human intervention in varying environments. For example, tests that utilize NDbUnit are easily run in continuous integration environments, such as CC.NET.

You may have heard of Continuous Integration (CI) for .NET, but perhaps thought it would take too long to install and configure the tools. By integrating the CC.NET, NAnt, and Subversion tools, you can quickly create a robust CI solution. In this entry, I walk you through the setup and configuration of these tools, step by step.

There are many ways to define CI, but just think of it as a process of integrating your software when a change is applied to your SCM repository. This may include compiling, inspecting, testing, deploying, and generating feedback.

Prerequisites
You need to install and configure IIS and have access to a Subversion repository. If you’d like a Subversion project to use for this example, use http://www.qualitylabs.org/svn/ndbunit/. Note: You will need an account established at qualitylabs.org.

Step 1 - Download and install tools
Subversion is the SCM tool used to manage your source code. Extract the Win32 version of the Subversion zip (svn-win32…) file to a designated directory
NAnt is the build scripting tool to build your software. Extract the NAnt zip file to a designated directory
CC.NET is the CI tool to run your build script whenever a change occurs in your SCM. Save and run the installation .exe file. CC.NET should be installed on a separate machine than the developer machines. However, for the purposes of this example, you will be installing CC.NET on the same machine.


Step 2 - Configure Subversion
Create a Windows environment variable called SVN_HOME. The value should equal the location of the Subversion installation directory from step 1. Add %SVN_HOME%\bin; to the System PATH environment variable.

Step 3 - Configure NAnt

Step 4 - Create NAnt build file
Create the following and save as default.build.

<?xml version="1.0"?>
<project name="HelloWorld" default="build">
  <target name="init" depends="clean" />
  <target name="clean" />
  <target name="checkout"/>
  <target name="compile"/>
  <target name="deploy"/>
  <target name="test"/>
  <target name="inspect"/>
  <target name="build" depends="init, checkout">
    <call target="compile" />
    <call target="inspect" />
    <call target="test" />
    <call target="deploy" />
  </target>
</project>

Step 5 - Configure CC.NET
Go to the directory where you installed CC.NET in Step 1. From here, go to the server subdirectory. In this directory there is a file called ccnet.config. Open this file and add the following:

<cruisecontrol>
  <project name="HelloWorld">
  <schedule type="schedule" timeout="60000"/>
  <modificationDelay>10000</modificationDelay>
  <sourcecontrol type="svn">
    <trunkUrl>http://www.qualitylabs.org/svn/ndbunit/</trunkUrl>
    <executable>C:/dev/testearly/svn-win32-1.3.1/bin/svn.exe</executable>
    <username>george.washington</username>
    <password>M@rtha</password>
    <workingDirectory></workingDirectory>
    <artifactDirectory></artifactDirectory>
    <autoGetSource>true</autoGetSource>
  </sourcecontrol>
  <build type="nant">
    <executable>C:/dev/testearly/nant-0.85-rc3/bin/nant.exe</executable>
    <baseDirectory>C:/dev/testearly/</baseDirectory>
    <buildFile>default.build</buildFile>
    <targetList>
      <target>build</target>
    </targetList>
    <buildTimeout>300000</buildTimeout>
  </build>
  <publishers>
  <xmllogger>
    <logDir>.websitelog</logDir>
  </xmllogger>
  </publishers>
  </project>
</cruisecontrol>

Update the trunkUrl attribute to the location of your Subversion server and project. Open a web browser and type:
http://localhost/ccnet/
The following should be displayed:

Troubleshooting
You may get tripped up with a few problems along the way. Here’s a list of some of some of the issues along with some troubleshooting techniques:

Conclusion
Of course, this is a very basic solution of integrating these tools. You’ll want to expand the NAnt build script to provide meaningful activities such as running automated tests and inspections and deploying the software. If you need to use a different SCM, you can simply change the source control configuration block in ccnet.config. By integrating your software upon any material source code change, you can better ensure that you will have working software by running through consistent, repeatable steps that compile, test, inspect, and deploy your software (into a development/testing environment). Integrate early. Integrate often.