There are manifold ways to manage 3rd party library dependencies within a build, ranging from simply checking them into your SCM system to Maven’s innovative on demand downloading of versioned libraries at build time. Each strategy has its own pros and cons. For example, keeping 3rd party libraries in a lib directory hanging of the root directory of a project is fine so long as that’s the only project within an SCM system– if you have multiple projects that all have lib directories and each has the same copy of various dependencies, you end up with copies of binaries across the system, which always seemed somewhat inelegant to me.
Of course, Maven’s auto-downloading of dependencies requires that you use Maven as the core build system (actually, the Maven team recently released a set of Ant tags that simulate dependency management), which is a problem if you have already invested in Ant. All’s not lost, however, as Ant users have the option of using Ivy, which is an excellent dependency management framework that works much like Maven.
Getting up and running with Ivy is quite easy too– it takes, at a minimum, 4.2 steps (and as you’ll see, you can get by with 3.1). Central to Ivy’s behavior is the notion of dependencies, which you define in an XML file. In this file you explicitly state the dependency’s name and version. This is key too– not only do you specify that your build requires JUnit compile (and run), for example, but that you are explicitly bound to version 3.8.1 (as opposed to 4.0).
As an example, here is a dependency file (named ivy.xml), which defines a project’s required libraries (which either are required for compilation or are runtime dependencies). By the way, Ivy is sophisticated enough to allow you to delineate specific types of dependencies– either runtime, compile, etc.
As you can see, this file makes no reference of where these files exist and where they’ll be stored– it just defines required dependencies for a particular project. By default, Ivy will scan Ibiblio for these jar files using the same exact pattern that Maven 1 utilized (name/jars/jarname-version.jar)
Secondly, if you’d like to use a custom repository from which project dependencies are downloaded (for example, if you don’t want to rely on Ibiblio and would rather have an internal web server serving up files), you can define a configuration file. This file is commonly called ivyconfig.xml and allows you to, among other things, define a custom dependency repository.
As you can see, in the file above, an additional repository is defined (along with a directory layout and mapping scheme), plus, Ibiblio is defined as the second repository should the first repository fail to yield a download. You’ll also notice that this file supports keywords (such as artifact, revision, and ext) by which custom patterns can be created. Plus, Ivy supports additional resolvers (such as file systems) if you’d rather not use http.
Step two, as you can see, is also optional, should you rather not want to define a custom repository; however, relying on Ibiblio may not be entirely prudent should you require explicit control over your project’s dependencies.
Once these files have been defined, incorporating Ivy’s functionality into Ant is a breeze– in fact, it takes two steps (or just one if you don’t require a configuration file) plus two minor steps. The first minor step is to add Ivy’s jars into Ant’s classpath (which means placing them in your ANT_HOME/lib directly), second, you must add a specific namespace to your Ant build file’s project element as follows:
Next, you need to create a new target (or update an existing target) that invokes Ivy’s behavior. If you want to use a custom configuration file, then you’ll need to use the configure task and point it to the requisite file like so:
<ivy:configure file="./ivyconfig.xml" />
Lastly, you need to force Ivy to retrieve the dependencies you specified in step one via the retrieve tag.
<ivy:retrieve />
Upon invoking the retrieve task, Ivy will check a local cache (which you can define) for the required libraries– if they don’t exist, it’ll download them. As a side note, if you’d like those jar files to reside in a local directory of the project, provide a ivy.lib.dir property, and Ivy will copy the files specified in the ivy.xml file into this directory.
For example, this snippet of XML defines a classpath in Ant with the jars downloaded via Ivy:
As you can see, getting Ivy up and running requires 4.2 easy steps:
Create an ivy.xml file with your required dependencies
Create a configuration file pointing to a custom repository (optional)
Place Ivy’s libraries in Ant’s classpath
Add an Ivy namespace
Call the ivy:configure task (optional, see #2)
Call the ivy:retrieve task
If you’re using Ant and are looking for an easy way to declaratively specify your project’s dependencies, then have a look at Ivy– it couldn’t be easier to set up. Plus, once you start digging, you’ll see that this is quite a sophisticated management system (capable of transitive dependencies, for example).
So you want to make MSTest work on a .Net 2.0 projects and have CruiseControl.NET monitor everything for you. Well maybe “want” is a strong word. Maybe someone (your employer) told you to do so. It is possible, actually not even that hard. So here’s what I did.
My tools of choice here are MSBuild and MSTest. I agree Microsoft should distribute MSTest separately from the Visual Studios Team Edition (VSTS) for Testers. However Microsoft does not, so for now, the path of least resistance is to give Bill his pound of flesh and install the IDE on the build server. I actually installed the Visual Studios Team Suite on the build server that way I had all my favorite tools handy. The build server didn’t even whine.
I started off with an MSBuild file Build.xml that:
Compiles two solution files
Cleans out the TestResults directory
Runs the tests
Converts the code coverage to a XML file
Deploys the out put to a MSI file
The CCNet.config file:
Pulls source from SVN
Labels source
Runs a MSBuild task using the Build.xml from above
I’ll break it down step by step. First I build up an item group of solutions to compile. I have two solutions to deal with, SourceMonitorSummary.sln which depends on cslacs.sln. Next I execute a MSBuild task named “BuildAll” to compile the solutions listed in @(Project). That will compile cslacs.sln first and then SourceMonitorSummary.sln.
Next I delete all the files and directories in the TestResults folder. That guarantees that when the tests run I will have one and only one folder inside TestResults directory. This is to support the code coverage report. John Cunningham has a wonderful blog The Visual Studio Team System “off-road” code coverage experience on converting the code coverage data from a test run into an XML file. I have a simple version of John’s code running that navigates the TestResult directory tree to find and move the data.coverage file to the out directory and then creates a Coverage.xml file that is later merged into the CruiseControl.net build log for the project.
Now for the good part. I execute the Exec Command task named “RunTests” that runs MSTest.exe from the command line. The key here is to setup the test meta data ahead of time. You need a test list, in my case I created BuildTests. Use the “Test/Create New Test List…” menu options in VSTE for Testers. When MSTest runs it will create a test results file in the TestResults directory. The file has a .trx extension. It is an XML file that can be merged directly into the build log.
You can also enable code coverage found under “Test/Edit Test Run Configurations/Local Test Run”. Pick the artifacts that you want code coverage against.
After that I execute the MSTestCoverageToXML.exe which converts the code coverage data to an XML file. If you have not enable code coverage for the test configuartion skip this task.
Finally I run the CreateSetup.bat for deployment. I’m using a Microsoft WIX file that gathers the output from the compile and packages it into an MSI. We have a repeatable build that can be executed from a command line “MSBuild Build.xml”.
Lets look at the CCNet.config file now, and see what is needed there.
The interesting stuff happens in the MSBuild task. CruiseControl is executing exactly the same build that the developer executes locally. Note the custom logger in the <buildArgs>, this is required to convert the MSBuild output in to XML. Next CruiseControl will merge the output files that we created in the automated build into the build log, this will make the test results and code coverage output available to the build report. The *.trx file contains the test results and the Coverage.xml file holds the code coverage output.
Finally we have the dashboard.config changes that will display the MSTest output.
There is another great blog entry covering MSTest and CruiseControl .NET at BM Bloggers. In that blog Richard talks about MSTest and displaying the test results in the build report. I have added his MSTestCoverage.xsl to the <xslFileNames> and commented out the unitttest.xsl as I’m not using NUnit. For code coverage report, John Cunningham to the rescue, Team System “off-road” code coverage analysis and reporting experience has a great description of how to format the code coverage data using XSLT transformation.
I don’t think that I have done anything really new here, I’ve just pulled together information from several sources that allow MSTest to co-exist with CruiseControl.net. I’d would love to here about other peoples experences with MSBuild/MSTest in their environments.
Are you looking for an opportunity to hang out (i.e. drink some wine) with like minded peers and ponder the pros and cons of Continuous Integration? Does your schedule have any leeway in late June? If so, we’re excited to announce that on Thursday, June 28th, we’re going to host a round table discussion on Continuous Integration war stories at our offices in Reston, Virginia.
Please join us and share your experiences with implementing and maintaining CI systems over wine and cheese– we’ll talk about various tools, CI techniques, and hopefully even pontificate over the future of CI. Plus, we’ll be raffling off a copy of Paul Duvall’s veritable vade mecum on the subject of CI: Continuous Integration: Improving Software Quality and Reducing Risk.
Hudson is a relatively new CI server on the block; however, it has attracted a fair bit of attention due to its easy set up. In fact, you can literally get it up and running in two steps– first download it, then type java -jar hudson.war. This command fires up a web server from which you can configure Hudson to monitor projects in either CVS or Subversion.
I did find myself a bit stuck configuring a Subversion project as there wasn’t an intuitive manner in which to specify credentials. After poking around the web, I managed to find a posting related to this very issue, which provided the information I needed to fix things.
Simply put, go http://host name/scm/SubversionSCM/enterCredential and enter the appropriate information and you’ll be good to go.
For the record, after solving this minor annoyance, I had a Hudson CI instance polling a Java project in Subversion (and executing a full Ant build upon a change) within 20 minutes. It seems the attention this project is receiving is credible!
Who cares about software metrics? As a Java developer, I do. Measuring certain aspects of my code lets me quantify my schedule, work effort, product size, project status, and code quality. Oh, and then my project manager cares too. If I don’t measure my current status (number of classes, dependencies to other modules, complexity, code coverage, defects as well as code smells - yes, we all know when we have code smells) and use the data to improve my code and my future work estimates, those estimates will just be guesses. And my project manager expects a little better than guesses.
Metrics is an Eclipse plugin that helps me take a holistic view of various aspects of my code. I use it on a daily basis to see how my project grows and if there are any deviations from the reference architecture. The plugin lets me take a closer look at:
Number of Classes
Method Lines of Code
Number of Methods
Nested Block Depth - Is nice, I like!
Depth of Inheritance Tree - nobody wants to traverse a 16-node deep tree trying to find what they need. 1-5 maybe, but 16….too many levels…
Afferent Coupling, Efferent Coupling - yea, whatever… we saw this here…same story…
Cyclomatic Complexity - If my code is simpler than Paul’s does that make me look “simpler” too?
…and more…
Quick Tutorial
Download and start Eclipse 3.2
Navigate to Help | Software Updates | Find and Install
In the Package Explorer, right-click the solarsystem project and open the Properties dialog. Locate Metrics and check Enable Metrics
Rebuild solarsystem
Navigate to Window | Show View | Metrics | Metrics View
You are looking at a table of various metrics covering the solarsystem project
Feel free to dig down on each metric as it goes from source folder to package to class down to the method
Notice all metrics for this simple project are in blue and there are no exclamation marks in the source code indicating any problems. Let’s change that and make life difficult for ourselves.
Open the preferences page for Metrics and check Enable out-of-range warnings. Drill one level down to Safe Ranges, locate Nested Block Depth and enter 5 as the Max value. Similarly, enter 10 as the Max value for McCabe Cyclomatic Complexity
In a Java Perspective open PlanetUtil.java and add the following method:
public void thisIsAComplexMethod() {
if (true) {
if (false) {
if (true) {
if (false) {
if (true) {
System.out.println("This is a deeply nested if sequence.");
}
} else {
}
}
}
}
if (true) {
if (false) {
if (true) {
if (false) {
if (true) {
System.out.println("This is a deeply nested if sequence.");
}
} else {
}
}
}
}
}
Rebuild solarsystem and return to the Metrics View. You now see two red lines in the results table and an exclamation mark in the code editor. Clearly the new code failed the cyclomatic complexity and nested statements rules of Metrics and therefore was marked red. The cyclomatic complexity of the method is 11, while the if statements nest for 6 levels. Both values are out of the acceptable ranges we defined earlier.
Locate the Dependency Graph View icon in the Metrics View. It can be found on the upper right corner and it is denoted by four dots connected in a square. Click the icon to open the graphical dependency view of Metrics.
The packages in the project are denoted by boxes in different colors. Blue means normal relationship; red means cycles and orange denotes the selected node. There is also a circle in the middle, the tangle center, which displays the number of packages in a particular tangle and the longest walk. In general, the longer the walk the more layers are involved and therefore there may be refactoring opportunities. Similarly, the existence of red nodes and tangles indicates cycles in the package dependencies and that’s also something that may be avoided with a little care. Finally, every node and the tangle center provide right-click menus to drill-down for more details.
The reason for this tangle (cycle) is the JUnit test PlanetUtilTest, which is located in com.solarsystem.jupiter. Let’s do something about this.
In the Package Explorer, locate PlanetUtilTest, right-click and select Refactor | Move. Move the test into a new package com.solarsystem.test.jupiter.
Rebuild the project. Open the Metrics View. Click the graph icon. Everything should be blue. There should be no tangles. No tangle centers. Blue is better than red. Life is good…
Paul Duvall and Levent Gurses have posted a vidcast (video podcast) discussing Levent’s presentation at the Better Software Conference next week in Vegas. Levent’s talk covers Eclipse and how a carefully selected set of plugins (JDepend4Eclipse, Checkstyle, Coverlipse, PMD, and Metrics) can help you create better quality code.
If you can’t check out his live presentation in Vegas, simply have a watch below and let us know what you think. Enjoy the show!
What does dependency analysis mean to you? To me it mostly translates to the fact that Common Services (CS) packages should not depend on any of their clients. Does that happen in practice? More than enough to make the case for dependency analysis.
Just like the other tools in this series JDepend can be used in Eclipse or as part of a Continuous Integration cycle. I like to see what’s going on in terms of dependencies and other architectural metrics all the time, so I use both. JDepend calculates:
Total number of classes and interfaces - number of concrete and abstract classes (and interfaces) in the package is an indicator of the extensibility of the package. How many?
Afferent couplings (Ca) - number of other packages that depend upon classes within the package is an indicator of the package’s responsibility. Who depends on me?
Efferent couplings (Ce) - number of other packages that the classes in the package depend upon is an indicator of the package’s independence. Who do I depend on?
Abstractness (A) - the ratio of the number of abstract classes (and interfaces) in the analyzed package to the total number of classes in the analyzed package. The range for this metric is 0 to 1, with A=0 indicating a completely concrete package and A=1 indicating a completely abstract package. How abstract I am? If I change today, how many of my dependents have to change with me?
Instability (I) - the ratio of efferent coupling (Ce) to total coupling (Ce + Ca) such that I = Ce / (Ce + Ca). This metric is an indicator of the package’s resilience to change. The range for this metric is 0 to 1, with I=0 indicating a completely stable package and I=1 indicating a completely instable package. How stable am I? Do I change often?
Distance to Main Sequence (D) - the perpendicular distance of a package from the idealized line A + I = 1. This metric is an indicator of the package’s balance between abstractness and stability. A package squarely on the main sequence is optimally balanced with respect to its abstractness and stability. Ideal packages are either completely abstract and stable (x=0, y=1) or completely concrete and instable (x=1, y=0). The range for this metric is 0 to 1, with D=0 indicating a package that is coincident with the main sequence and D=1 indicating a package that is as far from the main sequence as possible.
Package dependency cycles - dependency cycles are reported along with the hierarchical paths of packages participating in package dependency cycles. If I depend on you and you depend on me, we have to work hard to adapt to each other’s changes, don’t we?
The JDepend4Eclipse plugin is an implementation of the JDepend tool. It follows the same principles and provides visual analysis inside Eclipse. One thing to note is that the plugin works on compiled java bytecode, so its important for your project to build. Related to this is the fact that if you’re following the same package/folder structure for your unit tests as your source code it might be a good idea to compile them into a separate output folder, separate all unit tests into another Eclipse project or just change the unit test package names (by appending the name test for example ), like I show below.
Quick Tutorial
Download and start Eclipse 3.2
Navigate to Help | Software Updates | Find and Install
In the Package Explorer, right-click the source folder src/main/java and select Run JDepend analysis
In the newly opened JDepend perspective click on com.solarsystem.jupiter. Notice how the dependencies, packages with cycles, efferent and afferent dependencies and the metrics views get populated
Notice in the Dependencies view that the com.solarsystem.jupiter package has 3 Concrete Classes (CC), 1 Abstract Class (AC), 4 Afferent Dependencies (Ca) and 7 Efferent Dependencies.(Ce) Also the Abstractness (A) is 0.25 while the Instability (I) is 0.63. Finally, the Distance (D) to the main sequence is 0.11 which is denoted by a green dot in the Metrics view. Also in this view, notice that the package is marked as having a cycle. This is because the unit test PlanetUtilTest resides in the same package. JDepend works on compiled java code and therefore it picks up everything in the target/classes folder, including any unit tests. There is plenty of theory behind the various metrics gathered by JDepend and they are beyond the scope of this quick tutorial. It suffices to say that for any given package the shorter the Distance (D) to the main sequence, the better. In the above case the com.solarsystem.jupiter is used by four other packages, thus Ca=4. The package depends on 7 other packages, including junit.framework, therefore the Ce=7.
Generally speaking cycles between packages indicate complex interdependencies and should be avoided. In this step, let’s eliminate the cycles in the project.
In the Package Explorer locate src/test/java anc right-click on PlanetUtilTest
Select Refactor | Move…
Create a new package com.solarsystem.test.jupiter and click OK
Return back to src/main/java. Right-click and select Run JDepend analysis. All cycles are gone. While we eliminated the cycles, notice also that the distance jumped to 33 from 11 and is now denoted by a black dot.
In Java Browsing perspective select File | New | Interface. Enter a name, IBogus, select the package com.solarsystem.jupiter and click OK. This is just an empty interface.
package com.solarsystem.jupiter;
public interface IBogus {
}
Build the project
In Package Explorer, right-click the source folder src/main/java and select Run JDepend analysis
Notice the Distance is now 16 and the dot turned green. But what did the IBogus interface add to the package? Nothing. It just fooled the tool. This is where special care must be taken. Sometimes emty or unused abstract classes can negatively impact the static analysis and make one belive that the package is in a better state than it actually is. My recommendation is to use other tools that detect unused classes and imports and get the whole system cleaned-up before each dependency analysis.
If you use Eclipse for day to day Java development take a deep breath and relax. You are neither the first nor the only one. In fact, you are part of a growing community of Java developers in the world. SD Times recently reported that two-thirds of the enterprise software developers—66.3 percent—use Eclipse. This adoption rate is a success for an open source project, albeit supported by a giant like IBM.
The agile developers know that at the end it’s often not the tool that makes the difference. It’s the fundamental engineering practices. When the discussion goes beyond the tools it suddenly opens so many possibilities that even the most staunch tools proponents/opponents can sit down and unite in a common strategy. Let’s take coding standards for example. We all know that there are tons of tools for static code analysis and early defect detection. The Agile teams have choices in this matter. They can implement a set of static code analysis as part of their Continuous Integration (CI) cycle and get great reports the instant a potential bug gets checked-in. They can also encourage individual developers to use Eclipse plugins for code analysis and perform the same checks before they commit the code.
Two major plugins for code analysis in Eclipse are PMD and Checkstyle. Of course, Eclipse itself is also catching-up and it is constantly increasing its own set of code style checks. A look at Window | Preferences | Java | Compiler | Errors/Warnings reveals that the number and depth of the built-in code checks is fairly substantial.
Nevertheless, in this blog entry I would like to show you a quick example of how to setup and use Checkstyle for Eclipse .
Despite the fact that I use PMD and Checkstyle (and others for that matter, like Findbugs) in my daily projects, I favor Checkstyle for the following reasons:
The integration of the tool to the environment seems to be tighter
The rule customization and management in Checkstyle is way easier than PMD. I like!
Checkstyle provides a few nifty features such as a pie chart depicting the distribution of problems
Checkstyle comes with a richer set of predefined rules
Better website documentation and easier learning curve
All and all Checkstyle makes my life easier, and especially after configuring a set of rules that make the sense for the project environment, it’s a workhorse! An added benefit of the custom rulesets is that they can be checked into the code repository, shared among the development team and reused in the CI static analysis phase. The latter can be achieved through integrating tools such as CruiseControl, Maven and Ant with the respective Checkstyle plugins. This brings a unified coding standard to the entire development team.
Quick Tutorial
Download and start Eclipse 3.2
Navigate to Help | Software Updates | Find and Install
public class CheckMeUp {
public void thisMethodShouldHaveAShorterAndCoolerName(String WHYALLCAPS, int g) {
if (true) {
}
else {
}
while (true) {
String x = "The white fox jumped over the red rabbit, which in turn jumped over my yellow submarine.";
String y = x + "...";
}
}
}
Build the project
In the Package explorer right-click on checkmeup and select Properties | Checkstyle
Check Checkstyle active for this project and select Sun Checks (Eclipse) - (Global) as the ruleset
Navigate to Window | Show view | Other…
Select Checkstyle | Checkstyle violations
Notice the 27 violations in this simple Java class
Now, let’s play with the rules. Assume that you do not like the Line contains a tab character rule. The sample class has 12 of it. Navigate to Window | Preferences | Checkstyle and Click on New…
Select Internal Configuration, enter a name and description. Click Import… and navigate to ECLIPSE_HOME/plugins/com.atlassw.tools.eclipse.checkstyle_x.x.x and select sun_checks_eclipse.xml. This will import all Sun Eclipse rules for the new custom ruleset. Also note that Checkstyle will create a new custom ruleset xml file under WORKSPACE/.metadata/.plugins/com.atlassw.tools.eclipse.checkstyle/internal_config__xxxxxxxxxxx.xml. This file can be added into the code repository and shared among the development team. Click OK.
Double click the new ruleset. Click Whitespace and uncheck the Tab Character rule
Back in the project properties, select the new custom rule as your configuration
Click OK. The tab warnings are gone.
In the Checkstyle violations view click the chart icon in the upper right corner. This will open the pie chart view shown below
