Do you categorize tests? Grouping developer tests into three categories (unit tests, component tests, and system tests) ensures end-to-end system soundness which can make all the difference in reducing software build durations.
The basic idea of test categorization is to segregate tests based on how long they take to run, and run the fast ones more often than the slow ones. This gives the benefit that you actually run the fast ones more often, i.e., you don’t not run the fast ones because they are lumped in with the slow ones, and the whole bundle of tests is taking too long. Categorizing tests is easy with TestNG, using annotations. With JUnit it can be done with naming conventions or directory partitioning.
Earlier this month over at FTPOnline.com, David Falkenstein posted a blog - “I’m not in denial, I’m in a hurry” on static code analysis. There are several testing techniques for conducting code reviews, but his message is pretty clear-cut: static analysis tools are extremely effective in discovering bad practices and metrics about our code.
It’s important to note that static analysis alone is not a silver bullet solution. It is, however, a great addition to the QA toolbox and an effective complement to managing the quality of software. Keep in mind, (as I’ve said before), automated and manual code analysis methods are most effective when used in conjunction with each other - there is no replacement for a little human intellect.
Eric Sink recently wrote an article advocating the use of code coverage; where he has managed to achieve 100% coverage. I’m a huge believer that coverage data is quite valuable in the testing process, but is 100% such a good thing?
High coverage rates simply mean that a lot of code was exercised. High coverage rates do not imply that code was exercised well.
A common misconception of code coverage is that it is the final point of your quality development. Steve McConnell advises in his book, Code Complete, that “testing by itself does not improve software quality. Test results [and code coverage] are an indicator of quality, but in and of themselves, they don’t improve it. Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often.” Sound familiar?
Code coverage tools work as supplements, not substitues, for overall code quality. 100% code coverage certainly does not mean 100% tested. In the end, coverage measure is just an indicator of how various parts of your program were exercised - but gives you little confidence in your overall picture of quality.
Developer testing of J2ME application can be quite challenging due to the limited nature of the operating environment. I’ve run across two interesting resources for those interested in J2ME developer testing techniques:
Ask any development manager how their coding practices stand up against “average” and they will likely say, “Sure, there are things we could improve, but we are as good as (or better than) average.”
This response is known as the Lake Wobegon effect, after the fictional town in the Garrison Keillor radio series A Prairie Home Companion where, he opined, “all the children are above average.”
A 2005 article in the New York Times found when investigating an online dating service that only one percent (1%) of the population admitted to having “less than average” looks.
Funny and interesting, but what does it mean about your code?
Psychologists tell us that, especially in areas we consider important, we have poor skills at “comparative ability.” If tasks are important to us - whether we consider them easy or difficult – we respond with empirically illogical numbers: 75% - 90% of us claim to be “above average”.
This, of course, is especially true with subjective measurements. Asking someone how good-looking they are is a different exercise than asking how their weight compares with others who are the same height – especially if you first provide them with a scale and an actuarial chart of average weights.
As we self-evaluate our own capabilities in the important area of code (and code quality), we must recognize our own vulnerability to the Lake Wobegon effect. A great way to accomplish this is to move from the subjective to the objective.
Analysis of cyclomatic complexity, code duplication, code dependency and test coverage are objective measures that can provide meaningful, actionable information about how well our coding practices actually work. Without an objective score, you cannot measure your progress. And, as Bill Hewlett (HP) stated, “You cannot manage what you cannot measure.”
Once you have objective measures, seek out comparables within your organization and/or your industry to see how you truly fare against relevant averages.
But the real value is not in actual comparables, it is in avoiding the negative impact of our own common weakness of comparative ability and transforming our self-analysis from the subjective world to objective measure.
For the record, I don’t think you’re ugly. And, for those of you who are even more offended by the latter statement (you know who you are), I don’t think your code is ugly either.
ComputerWorld published an article entitled “Cobol Coders: Going, Going, Gone?” in which they report that “62% of the [352] respondents reported that they actively use Cobol.” Moreover, of that group, “25% said they’d like to replace Cobol with something else but have found that too difficult or too expensive.”
Clearly, there is a lot of Cobol still running out there– somewhere. This is a nice tidbit of information to keep in the back of your mind as you evaluate your own applications– how long do you think they’ll last? The big secret of the software world is that most software stays around much, much longer than we ever imagined it would.
Yet, most of the maintenance issues that a project deals with during its lifetime are injected in early stages. In fact, one of the only ways to stay ahead of the game is through visibility. Increasing visibility enables teams to understand:
Who changed what and why?
How is complexity trending?
What’s tested and what isn’t?
Where are the hot spots for defects?
How difficult is it to change and how long does it take to deliver?
Note, the ability to answer these questions implies a number of things:
A robust CM system (and associated process) is in place
Software builds are repeatable
There is a process to report metrics
Traceability exists for defect correlation
There are automated tests and associated coverage metrics
Once those questions can be confidently answered, teams can make changes with an understanding of the risks because they can quantify quality. The question of porting to another platform (or a rewrite) isn’t blindly answered with it’s “too difficult or too expensive” but its answered with data to back it up.
Like many, I’m a long-time reader of the very insightful “Creating Passionate Users” weblog. Most of the articles are authored by Kathy Sierra, a veteran in the technical publishing industry who has an eclectic background in software and web development. Kathy is also the founder of one of the largest community web sites in the world, javaranch.com.
Smelly code will surely cost you valuable time. How much time do you spend maintaining project build scripts? Probably much more than you’d expect or would like to admit.
The third installment in IBM developerWorks’ series “Automation for the people” demonstrates how to improve a number of common build practices that prevent teams from creating consistent, repeatable, and maintainable builds. Check out “Remove the smell from your build scripts” and learn more about the nine most common build smells you should avoid, why you should avoid them, and then how to fix them.
JavaRanch published an article some time ago entitled “Evil Unit Tests” in which the author shares a story where a client’s build took 45 minutes to run and tests would frequently break because they:
fell into the popular trap of thinking that because they run the tests with JUnit, they must be unit tests. 90% of their problems could have been resolved with just a wee bit of vocabulary.
Bingo! Well said (or written)! He goes on to define a series of test types, which are largely similar to definitions we’ve recommended before. The most interesting point made is regarding how developer tests differ from Functional tests in how they isolate aspects of code– simply put, a developer test can pin point sections of code in isolation whereas a higher level test must traverse the system to reach a target.
If you are outsourcing development or quality services to India, consider that even top companies like Google are struggling to hire talented engineers there. Two slightly different versions of an article describing the challenges can be found on the InformationWeek and Gulf Times websites.
If high-value companies like Google are struggling, you would expect outsourcing companies to have even more difficulty attracting talent.
