Home | About Us | Stelligent  

TestEarly Weblog

Build Management

Build Management and Continuous Integration and Publications and /Duvall11 Nov 2008 04:41 pm

It can be a significant burden to manually apply upgrades to an existing database. Moreover, it’s often an error-laden activity. IBM developerWorks’ recently published an article on this topic for Paul Duvall’s (Stelligent CTO) popular Automation for the people series called Hands-free database migration.

Databases are often out of sync with the applications they support, and getting the database and data into a known state is a significant challenge to manage. Learn how the open source LiquiBase database-migration tool can reduce the pain of managing the constant of change with databases and applications.

Developer Testing and Build Management and Tutorial and Agile and /Duvall23 Sep 2008 07:02 am

gant-imageYes, this is exactly what I am frequently asked by my clients and many developers. It isn’t easy to answer this question. There are several projects using Ant. Should you run away from Ant just because there is a new cool tool out there called Gant? Should you switch to Gant just because you dislike XML? Not at all. Let’s take a closer look and see what might make you switch to Gant.

When to choose Gant?

1. Complicated Build Files. If your ant build files are becoming too complicated, and hard to manage, it’s time to see if using Gant can help. Let me explain what I mean by complicated build files. If you have too much of conditional logic within your build files, say something similar on the lines shown below in Listing 1:

Code Listing 1:


    <if>
      <isset property="sqlserver"/>
      <then>
        <do something here/>
      </then>
    </if>
    <if>
      <isset property="oracle"/>
      <then>
        <do something else here/>
      </then>
    </if>
    <if>
      <isset property="derby"/>
      <then>
       <do something for derby here/>
      </then>
    </if>
    <if>
      <isset property="db2"/>
      <then>
        <do something for db2 here/>
      </then>
    </if>

Or even something like this where you might be supporting deployment to different application servers based on some property in your build.properties or by the user as shown in listing 2.

Code Listing 2:

    <if>
      <isset property="server.jboss4"/>
      <then>
        <deploy to JBoss 4/>
      </then>
      <elseif>
        <isset property="server.weblogic10"/>
        <then>
	<deploy to web logic/>
        </then>
      </elseif>
      <elseif>
        <isset property="server.glassfish"/>
        <then>
          <deploy to GlassFish/>
        </then>
      </elseif>
      <elseif>
        <isset property="server.someother version"/>
        <then>
          <deploy to this some other version/>
        </then>
      </elseif>
    </if>

Things get out of hand when you have conditional logic as shown above in your build scripts. The listings I have are just the skeleton, imagine what happens when we start adding the actual deployment logic for all these application servers. It doesn’t matter how you refactor this, it is still going to be very complicated. Trust me, I have written build scripts which were several thousand lines, and refactoring them was not a trivial task.

2. Custom Ant Tasks. I myself am guilty of writing many of these. There are many situations which arise in projects where we create custom ant tasks. It is simple once you know how to write one, and than for every complicated task you need to perform, you involuntarily will start writing custom ant tasks.

Anyone writing a custom ant task will:
a. Create a new class that extends Ant’s org.apache.tools.ant.Task class.
b. For each attribute, write a setter method.
c. Write an execute()method that does what you want this task to do.

There isn’t anything wrong in doing the above, but imagine each time you want to make a small change you will have to make changes within your Java source code, compile, test, re-package.

3. Scripting. You can extend Ant further by not writing custom ant tasks, but by using small snippets of code written in an interpreted language like JRuby, BeanShell, or Groovy. These code snippets can be placed within your build files or in separate text files. If you are using Groovy’s Ant task, your build file might look something like this:

 <groovy classpathref="build.classpath">
  import some.package
  import another.package
  def fullpath = "${.basedir}/${defaulttargetdir}"
   def somefile = new SomeFile(projectName:"${pname}",
     buildLabel:"${label}", buildTime:"${new Date()}")
   def xml = "${fullpath}/dashboard.xml"
   new File(path).write(somefile.generateReport())
   ant.xslt(in:path,
    out:"${properties.defaulttargetdir}/some.html",
    style:"${properties.defaulttargetdir}/lib/report-style.xsl")
 </groovy>

Imagine having several lines of XML in your build files which have many of these small snippets of scripts. I myself don’t like mixing and matching build files with code snippets. If you have a team where everyone is in the same page, everything works fine. What if a team member has no clue about any of the Scripting languages? He/She will have no clue how to make minor changes when things go badly. If you have all the above or even one of the above three cases, you seriously need to consider using Gant. To quote Aristotle:

For the things we have to learn before we can do them, we learn by doing them.

So, lets see how easy it is to learn Gant and see how things can improve.

This part covers some very basics of Gant. The next part, will dive deeper into Gant by using it with a sample project to build our application, and we will also see how to use it with our CI Server.

What’s Gant?

Gant is a build tool that uses both Groovy and Ant. With Gant, you describe your build process using Groovy scripts. Stated simply, Gant allows you to specify the build logic using Groovy instead of XML. The next thing you may ask is ” Is Gant a competitor to Ant?”. Let me quote from the Gant web site to make things more clear here :

Whilst it might be seen as a competitor to Ant, Gant uses Ant tasks for many of the actions, so Gant is really an alternative way of doing builds using Ant, but using a programming language rather than XML to specify the build rules.

Download and Install Gant.

In order for Gant to work, you should have Groovy installed. You can download and follow the installation instructions for Groovy here.
As I said earlier also, in order to use Gant, you should have knowledge of Groovy as well. If you have never written Groovy code before, there are many interesting books on Groovy like:

a. Groovy in Action

b. Groovy Recipes

c. Groovy Refcardz

You can also read the getting started guide on the Groovy web site, which should give you a good starting point.

Download the latest version of Gant from here. Gant is currently at version 1.4.0. Unzip it to a folder. If you already have your GROOVY_HOME set, that’s all you need to use Gant.

Getting Started.

Open a console, and type gant. You should see a message as shown below:

meera-subbaraos-macbook-9:~ meerasubbarao$ gant
Cannot open file build.gant
meera-subbaraos-macbook-9:~ meerasubbarao$

You are all set at this point to use Gant in your projects.

Help Information: Open a console, and type gant -h. This will provide you with all the necessary help information you need as shown below:

meera-subbaraos-macbook-9:CodeMetricsProject meerasubbarao$ gant -h
usage: gant [option]* [target]*
-c,–usecache Whether to cache the generated class and

perform modified checks on the file before re-compilation.
-n,–dry-run Do not actually action any tasks.
-C,–cachedir The directory where to cache generated
classes to.
-D = Define to have value .
Creates a variable named for use in the scripts and a property
named for the Ant tasks.
-L,–lib
Add a directory to search for jars and
classes.
-P,–classpath
Specify a path to search for jars and
classes.
-T,–targets Print out a list of the possible targets.
-V,–version Print the version number and exit.
-d,–debug Print debug levels of information.
-f,–file Use the named build file instead of the
default, build.gant.
-h,–help Print out this message.
-l,–gantlib
A directory that contains classes to be used
as extra Gant modules,
-p,–projecthelp Print out a list of the possible targets.
-q,–quiet Do not print out much when executing.
-s,–silent Print out nothing when executing.
-v,–verbose Print lots of extra information.

Create a new file called build.gant at the root of your project. gant-project

Did a similarity between Ant and Gant strike you here? Ant build files are usually called build.xml, and they are created as a common practice within the root of your project folder as well.

If you have written or even modified Ant build files, you will know that it contains one project element, which in turn contains a name,the default target and the base directory.

Code Listing 4:

<project name="GantSamples" basedir="." default="compile">

So for example, sayHello target in Ant would look something like this:

<target name="sayHello" description="Saying Hello">
	<echo message="Hello from Stelligent"/>
</target>

Lets create the sayHello target, and also see how to set it as the default target in Gant as well.

A Gant target has a name and a description:

Code Listing 5:
target ( target-name : target-description ) {
groovy code sequence
}

The above sayHello target in Gant would translate as shown below:
target(sayHello:"Saying hello"){
Ant.echo(message:"Hello from Stelligent")
}

Now, open a command window and type gant at the root of the project where the build.gant file exists. You should be able to see a output like:

meera-subbaraos-macbook-9:CodeMetricsProject meerasubbarao$ gant
Target default does not exist.

Gant is complaining that we haven’t set a Default target. Lets see how to do the same:

Default target: Within Ant, you define the default target from within the project element as seen in Listing 4. The default target is the target called if no target is specified from the command line. There however is no project tag within Gant. There are two ways of specifying the default target as shown below

1. You simply create a target whose name is default.
target ( 'default' , 'The default target.' ) { aTarget ( )
2. or even simply:
setDefaultTarget ( aTarget )

In order to get our sayHello target working, we need to add one of the above scirpts to our build.gant file.

Code Listing 6:
setDefaultTarget(sayHello)
or
target ("default": "The default target." ) {
sayHello ( )
}

Complete listing of build.gant:
target(sayHello:"Saying hello"){
Ant.echo(message:"Hello from Stelligent")
}
/*
target ("default": "The default target." ) {
sayHello ( )
}
*/
setDefaultTarget(sayHello)

And you should be able to see:

meera-subbaraos-macbook-9:CodeMetricsProject meerasubbarao$ gant
[echo] Hello from Stelligent
meera-subbaraos-macbook-9:CodeMetricsProject meerasubbarao$

That was easy! If you have build files and it is becoming unmanageable by your team, there is a tool out there which can convert your Ant scripts to Gant scripts as well. I haven’t used it, but you can try it here.

In this part of the series, we learned when to move over from Ant to Gant, downloaded and installed Gant, and finally wrote a simple gant build file. In the next part of this series, we will see Gant in Action within a simple Java project.

And as always, keep us posted here if you are encountering any problems getting started with Gant. Stay tuned.

Developer Testing and Build Management and Continuous Integration29 Jul 2008 09:59 am

ThoughtWorks, announced the general release of Cruise, a continuous integration and release management system. ThoughtWorks, are also creators of the open source CI Server CruiseControl. Cruise introduces the powerful concept of pipelines, which makes it simple to manage and test changes in an application from check-in through deployment.

ThoughtWorks is offering a free 30-day trial of Cruise (on Windows, Mac OS X and Linux), following which it will remain free to use for teams requiring two or fewer software agents. In addition, Cruise will be available at reduced or no cost for most open source projects, academic institutions and non-profit organizations.

For more details visit:

1. Cruise

Developer Testing and Build Management and Continuous Integration25 Jul 2008 08:50 am

An automated build process with Ant is one of the most crucial things required in any CI process. Ant is the build tool of choice for many Enterprise Java projects. This is an XML file, usually called build.xml, which describes a project’s dependencies. At the beginning of any project, this build file will be somewhere around 70-80 lines, which would include targets for compiling source and tests, running these tests, creating the libraries, and so on… But, as the project grows, this file grows as well and becomes as huge as several thousand lines; which is complicated and hard to maintain.

At this point, you might decide to refactor the build file, but before you being to do so, it would really help if you had a bigger picture of this several thousand lines of XML file, right? As the proverb goes “A picture is worth a thousand words,” I will show you how this one picture we are just going to create shows several lines of XML files.

1. Download the libraries.
I have tested diagrams from both Grand as well Vizant. I didn’t see any significant difference in either one of them. So, based on your preference and the license, download the libraries from here:

  • Grand
  • Vizant
  • Graphviz The dot file needs to be post-processed with Graphviz to produce the actual graph.

2. Create an Ant Target.
I have two listings show below here; one for Grand and one for Vizant. If you have used custom ant tasks, the listing shown below is self explanatory. Before using either Grand or Vizant, define a task. Next, give the name of the build file, its location, and the output. To view the graph you need to transform the dot file into something else using the dot command. In our case, we are generating a PDF file, but you can generate any other format as well; like JPG, PNG and so on.

Listing 1 for Grand:

<target name="grandAntDiagram">
    <typedef resource="net/ggtools/grand/antlib.xml" classpath="${basedir}/lib/grand-1.8.jar" />
        <grand output="build.dot" buildfile="${basedir}/build.xml" />
		<exec executable="dot">
		  <arg line="-Tpdf -Gsize=11.69,8.27 -Grotate=90 -o build.pdf ${basedir}/build.dot" />
		</exec>
</target>

Listing 2 for Vizant:

<taskdef name="vizant" classname="net.sourceforge.vizant.Vizant" classpath="${basedir}/lib//vizant-0.1.2.jar" />
<target name="vizantAntDiagram">
		<vizant antfile="${basedir}/build.xml" outfile="vizant-build.dot">
			<attrstmt type="graph">
				<attr name="ranksep" value="1.2" />
				<attr name="nodesep" value="0.5" />
			</attrstmt>
		</vizant>
		<exec executable="dot">
			<arg line="-Tjpg vizant-build.dot -o build.jpg" />
		</exec>
</target>


3. Run the Target.

You can run the targets from the command lie, from the IDE of your choice or from your CI server by just calling the above targets.

mobile-166-217-041-119:webservices-samples meerasubbarao$ ls
build.dot		dist			run.bat
build.pdf		instrumented		src
build.properties	lib			test
build.xml		passfile		vizant-build.dot
classes			qalab.xml
cobertura.ser		reports
mobile-166-217-041-119:webservices-samples meerasubbarao$ ant grandAntDiagram
Buildfile: build.xml

grandAntDiagram:
    [grand] Loading project /Users/meerasubbarao/Development/webservices-samples/build.xml
    [grand] Setting up filter chain
    [grand] Writing output to /Users/meerasubbarao/Development/webservices-samples/build.dot
    [grand] Outputing to /Users/meerasubbarao/Development/webservices-samples/build.dot

BUILD SUCCESSFUL
Total time: 0 seconds
mobile-166-217-041-119:webservices-samples meerasubbarao$

Here are some sample diagrams for the build files I had.

1. A simple one, this was the build file I used for my article I wrote for Javalobby.

simple-grand-diagram

2. And, this one which was generated by an IDE for another article I was working on. Does your build file look like this? If it does, than it is time to refactor the build files heavily.

ide-generated-build-file

P.S: The listings shown above in itself are sufficient for you to get the diagrams working in your build files, the one thing you will need to change is the location of the libraries. Also, don’t forget to download and install Graphviz, if not you will get an exception like this:

BUILD FAILED
/Users/meerasubbarao/Development/webservices-samples/build.xml:107:
      Execute failed: java.io.IOException: dot: not found
Total time: 269 milliseconds
Developer Testing and Build Management and Continuous Integration16 Jul 2008 03:25 pm

Consultant Eric Minick says that CI should be focused on testing, not on builds.

He makes the following interesting points:

  • automated tests verify successful integration, not the compilation/build process
  • running a build multiple times with different test “suites” is complicated and error prone
  • performing many different kinds of builds in an enterprise environment is wasteful and much of the testing is outside of source control

His recommendation is that a given software “build” (a set of packaged software bits that represent the output of the compile/generation process) should be passed through multiple testing stages over time. Personally, I visualize this like a car going through a car wash - we don’t build a new car for each brush and spray hose - we use the same car over time.

My take: I think, conceptually, that there’s a lot of merit here - because if you have an active project with a lot of commits, and you’re firing off all sorts of different builds with different test profiles, you are dealing with a constantly changing code-base as you run different types of tests. In other words - the code that you run the “database” integration tests on might be subtly different from the code you run the “web services” integration tests on.

And that’s a fair point - there are situations where you end up with a bug that “slides between” two types of builds. Eric’s solution certainly addresses that issue, in a reasonably elegant way.

However:
A) those issues don’t happen that often, in my experience (if they happen all the time to you, then definitely consider Eric’s idea)
B) Most sites I know of have at least one “Mother of all Test Suites” to cover and catch bugs that slip past the “lesser” test suites.
c) The amount of “waste” involved in fairly meaningless

I applaud UrbanCode’s innovative thinking. If this is something that speaks to you, by all means, give it a try. But don’t trash your existing build model just yet.

Developer Testing and Build Management14 Jun 2008 07:40 am

As I mentioned in an earlier post entitled “Three commands for Subversion enlightenment“, Subversion’s diff command is quite helpful in determining what local changes have been made.

The output of the diff command is somewhat verbose in that it literally compares the local version to what’s in Subversion. If you’ve made a lot of changes, the output can be overwhelming.

For instance, running the diff command (for a sandbox where only one file has changed) yields output like so:

aglover$ svn diff
Index: test.properties
===================================================================
--- test.properties     (revision 126)
+++ test.properties     (working copy)
@@ -1,5 +1,5 @@
 #HSQL Database Engine
-#Tue Jun 03 10:49:01 EDT 2008
+#Thu Jun 12 17:04:23 EDT 2008
 hsqldb.script_format=0
 runtime.gc_interval=0
 sql.enforce_strict_size=false

In truth, most of the time, I just want to know what has changed not necessarily how. Accordingly, you can pipe the output of diff to a grep-like utility searching for the line containing the word “Index:” like so:

%>svn diff | egrep "Index:"

In my case, I’m using egrep to search for a line of text containing the keyword. If, for some reason, the files you are changing actually contain that word, you might want to precede the phrase with a ^ (which matches the starting position of a line).

Consequently, using this pipe filter yields the following output:

aglover$ svn diff | egrep "Index:"
Index: test.properties

Now I can more easily understand what files have changed and update a repository appropriately.

Developer Testing and Build Management06 May 2008 08:08 pm

Mastering the art of command line jujitsu is something every developer (regardless of OS) should strive for. While OS UIs are becoming more and more slick and helpful tools are abundant, possessing the ability to drop into a console and efficiently knock something out (find . -iname *.java | xargs egrep -i “todo” anyone?) has come in handy, for me at least, on more than one occasion.

For instance, every modern IDE now has a plug-in that facilitates working with Subversion– in fact, when I made the switch to Subversion from CVS a fews years back, I initially failed to even learn the Subversion commands– my favorite tools did it all for me graphically! Yet, in some cases I found myself distrusting the implementations these tools provided– for instance, one of my favorite CVS commands was cvs -nq update, which effectively tells you what’s changed in your local sandbox as compared to the tip of the CVS repository. This, of course, is handy in figuring out what you plan on committing; hence, I would run this command before a cvs commit so as to filter out unneeded changes.

This developmental cadence of making changes and checking them against the tip (and consequently committing them) is actually common among SCM systems– the cadence has three steps:

  • updating a local sandbox to reflect what is in a repository
  • understanding what has changed locally
  • committing local changes into a repository

With Subversion, these three steps can be achieved via the command line with three commands:

The svn update command takes any changes in a Subversion repository and pushes down to a local sandbox. For instance, if more than one person is making changes and committing those changes into a repository, when another person runs the svn update command, those changes will be literally downloaded into that person’s sandbox.

Using the update command is as easy as opening up a console and typing:

tty:~/dev/projects/easyb/ yts$ svn update

If there are changes in the repository, you should see some scrolling text like so:

U    maven-plugins/maven-easyb-plugin/src/main/java/org/easyb/maven/StoryReportMojo.java
U    maven-plugins/maven-easyb-plugin/src/site/apt/usage.apt
U    maven-plugins/maven-easyb-plugin/src/it/story-report-location-test/pom.xml
U    maven-plugins/maven-easyb-plugin/src/it/site-examples-test/pom.xml
U    intellij-plugins/easyb-plugin/easyb-plugin.iws
U    intellij-plugins/easyb-plugin/src/main/java/org/easyb/plugin/ui/EasybPresenter.java
U    intellij-plugins/easyb-plugin/pom.xml
Updated to revision 329.

Note, the U means updated– there are other characters for new files added, merged, in conflict, etc. What’s key here is that by running the update command, your local sandbox is in sync with what’s on the server. Running this command often often means you’ll probably avoid conflicts too!

After you’ve made some local changes, it’s often times helpful to capture what those changes were– if you’ve been working for any length of time or working with more than one file, you’ll most likely find the diff command indispensable.

Like the update command, diff can be run via the command line like so:

tty:~/dev/projects/easyb/ yts$ svn diff

This command will compare what’s in the repository with what’s in a local sandbox; what’s more, it’ll report on the exact changes as follows:

Index: behavior/groovy/org/disco/bdd/prototype/FixtureFeature.story
===================================================================
--- behavior/groovy/org/disco/bdd/prototype/FixtureFeature.story        (revision 322)
+++ behavior/groovy/org/disco/bdd/prototype/FixtureFeature.story        (working copy)
@@ -4,7 +4,7 @@
 description "this story is fleshing out fixture logic for scenarios"

-each "some description" , {
+every "some description" , {
     given "a value", {
         value = 12
     }
Index: src/groovy/org/disco/easyb/StoryBinding.groovy
===================================================================
--- src/groovy/org/disco/easyb/StoryBinding.groovy      (revision 322)
+++ src/groovy/org/disco/easyb/StoryBinding.groovy      (working copy)
@@ -102,7 +102,7 @@
       listener.describeStep(description)
     }

-    binding.each = {  description = "", closure = {} ->
+    binding.every = {  description = "", closure = {} ->
         beforeScenario = closure
     }
     return binding

As you can see in the text above, the local code (labeled as “working copy”) has changed the each call with every, which appears in revision 322 of the repository.

Once you’ve reviewed these changes and are happy with what you’ve got, you can then proceed to push these changes into the repository via the svn commit command– of course, this assumes you’ve run a local build and it passed successfully, right?

tty:~/dev/projects/easyb/ yts$ svn commit -m "updated dsl to use every instead of each"

Note that the commit command can take a flag (-m) that enables you to specify a comment. You should see some text indicating the changes are being pushed into the repository.

Sending        easyb/behavior/groovy/org/disco/bdd/prototype/FixtureFeature.story
Sending        easyb/src/groovy/org/disco/easyb/StoryBinding.groovy
Transmitting file data ..
Committed revision 325.

Note how these changes increment the revision number in Subversion– this is key as all commands allow you to work with various revisions (via flags); hence, by knowing a particular revision, you can logically branch if needed.

These three Subversion commands will come in handy if you aren’t already using them; hence, learn them now! Indeed, for becoming one with the command line is the path to enlightenment.

Developer Testing and Build Management and Continuous Integration17 Apr 2008 12:23 pm

Let’s say you have a fairly complicated build structure, with multiple batch files that fire off different aspects of the build (even if, at the end of the day, they just call ant).

And let’s assume that for various historical reasons, these batch files encapsulate their environment variables using SETLOCAL and ENDLOCAL.

At first glance, this is a pretty cool way to scope out your environment variables. But then, you find out that Hudson determines if a build has succeeded or failed by looking at the %ERRORLEVEL% environment variable. If it is not 0, the build has failed.

Uh oh. Your build is failing, but the moment ENDLOCAL is called, %ERRORLEVEL% is set back to its previous value, which, most likely, will be 0.

Given this situation, how do you get the %ERRORLEVEL% back up to Hudson? Well, don’t call ENDLOCAL. Instead use:

EXIT %ERRORLEVEL%

at the end of your batch file. This works because there’s an implicit ENDLOCAL at the end of your batch file, and EXIT will return the ERRORLEVEL as the exit value.

Developer Testing and Build Management18 Feb 2008 11:30 am

It seems the build platform debate continues to heat up with Steven Devijver’s posting on the DZone entitled “What’s Wrong with Build Systems in Java Today?” where he previews a build platform that builds upon Ant and Maven 2 and adds, as he states:

Better-than-Maven 2 JAR and project dependency management, No XML, and Integration of Ant, Maven 2, Gant, Buildr, Rake, Grails, … projects in multi-project builds

Clearly, in the Java space, Ant is the de-facto build platform; consequently, the future of build languages will most likely need to leverage the extensive infrastructure in place to support Ant– dropping XML is key should you require a more imperative style of building things. Dependency management has always been a splintered solution ranging from Ivy to Maven’s declarative dependency framework to Ruby’s gems– all interesting and viable solutions, which makes his statement all the more interesting.

All in all, these are exciting times as dynamic languages are enabling a complete change in the way we define software builds, which will ultimately lead to tools that make building software (that is, compiling, testing, inspecting, deploying, etc) easier.

Developer Testing and Build Management and Continuous Integration and Agile04 Feb 2008 05:24 am

Production-ready software, on-demand

The general notion of the practice of Continuous Integration is that it’s performed during the “development cycle”. I think we need to expand this into areas typically thought of as “latter lifecycle” activities. In my experience, even the most Agile of projects aren’t considering all aspects of what it takes to make software “production ready”.

Over the past year, I’ve been advocating a much more intensive criteria for Continuous Integration with my peers and customers, which is more like “Continuous Production” meaning you can produce production-ready software on a daily or even a continuous basis. Effectively, we’re talking about extending the Continuous Integration model into all environments (Test, Stage and Production), not just the development environment. In fact, at Stelligent, our purpose is to get our customers to production-ready software on a daily basis.

A posting on Wikipedia describes Continuous Production as “a method used to manufacture, produce, or process any product without interruption. There is no discrete rate at which goods are produced, as opposed to a batch production process, or job production.” I’m suggesting we move toward a similar model for developing software.

continuous-production

A lot is bundled in this term, Continuous Production, though. In my experience, it means that you automate most everything it takes to create software so that it can be deployed into the user’s environment any time. It means you may automate some or all of the following:

  1. Compilation of source code and tests - this is a given for CI.Tools: Ant, MsBuild, rake, NAnt or similar build tool.
  2. Database Integration/Migration to target database server (e.g. Oracle) - Some shops are doing this with CI. Tools: Ant sql task or ORM-based Ant tasks, dbdeploy
  3. Testing - execution of unit, component, system, functional/acceptance, security, load and performance tests - Some shops are doing this with CI. Tools: JUnit, DbUnit, Selenium, JUnitPerf, FindBugs, JMeter, NUnit, NDbUnit
  4. Inspections - execution of static/dynamic analysis checkers: code coverage, code duplication checks, coding standards, cyclomatic complexity and dependency analysis - Some shops are doing this with CI. Tools: PMD, CheckStyle, FindBugs, JavaNCSS, JDepend, Cobertura, NCover, NProf, Source Monitor, NCover
  5. Deployment - Deployment and repeatable configuration to servers (e.g. JBoss) - Some shops are doing this with CI. Tools: Application Server-specific Ant tasks (e.g. WebLogic)
  6. Remote Deployments to target environments - For example, from a build machine to staging environment (for web containers and database servers). Tools: Ant, Java Secure Channel and SmartFrog, Capistrano
    1. You will need a way to rollback to a previous deployment (this includes database changes)
  7. (Optionally) Creation of installation distribution - Few shops are doing this with CI. Tools: NSIS and antinstaller, InstallShield .
  8. (Optionally) Generation of distribution media - Few shops are doing this with CI
  9. Source labeling and (possibly) build labeling - Some shops are doing this with CI
  10. Automated notification of build status - This is a given for CI via Email, X10 devices, etc.
  11. Automating the promotion through QA, Staging and Production environments - Promote the binary and accompanying artifact through each target environment. Tools: Build Management servers such as AntHill Pro are useful for build promotion
  12. Documentation Generating the readme docs, installation and user guides. Tools: JavaDoc, NDoc, Doxygen, GraphViz, Grand, etc.

Of course, the actual steps will vary depending the type of software application you are producing whether it be a rich client, web application, embedded or other…but you get the point. Continuous Production, to me, means you have the capability to create production-ready software whenever necessary .

What I am saying is that the creation of production-ready software should become a “nonevent “.

The benefits are numerous, but here are a few:

  • Decisions can be made based on working software , not on a task item on a project management chart - no more “it’s 80% done”
  • Eliminate the reliance on an individual with specialized knowledge to produce the software for production
  • There’s “done”, there’s “done, done” (coded with acceptance tests). With Continuous Production, it’s “done, done, done” and ready for release to users (code, acceptance tests, operating in the customer’s environment)

Note: This does not mean that you will be taking up cycles producing production-ready software with every change to the version control repository. Test, Stage and Production builds are more likely to be performed on demand (by a person). However, the point is that your build environments are capable of building production-ready software on a continuous basis.

There are very, very few projects that are doing this right now. I’d have to guess it’s much less than 1%. It makes me think of the saying “it’s not the destination, it’s the journey”. If your team sets this as a goal, the closer you get to it, the less you’ll be spending time on environment-related, repetitive and error-prone issues and the more time you spend on new features and improvements to the software. Is it possible, right now, for every project to do this? Probably not. However, once you start asking questions like “What do I need to do to make this feature production ready?”, it changes your perspective on everything from coding to testing and deployment.

In reality, I think even the most automated development environments still need (or should) have a manual component when creating software that is production-ready. This may include usability testing or even a simple manual sanity check. But, I think the more we move toward this model, the less headaches we’ll have and the better our estimates.

In my book on Continuous Integration, I make a reference to something Tim O’Reilly mentioned regarding Flickr’ s ability to deliver updates to its web software as frequently as every 30 minutes. This is only possible if you’ve automated many of the deployment steps.

Let me know if you or someone you know is doing this or moving toward this type of model. I’m curious to hear your experiences with it.

Next Page »