Three Questions About Each Bug You Find

I learnt this in my first job, Tango/04 in Barcelona, they really wanted to do the right things right. 

Three Questions About Each Bug You Find

Tom Van Vleck

Do you sometimes fix a bug, and then find another bug related to the first or to the way you fixed it? When I fix a bug, I ask myself three questions to make sure I've thought carefully about its significance. You can use these questions to improve productivity and quality every time you think you've found and fixed a bug.

The key idea behind these questions is that every bug is a symptom of an underlying process. You have to treat the symptoms, but if all you do is treat symptoms, you'll continue to see more symptoms forever. You need to find out what process produced the bug and change the process. The underlying process that caused your bug is probably non-random and can be controlled, once you identify what happened and what caused it to happen.

Before you ask the three questions, you need to overcome your natural resistance to looking carefully at the bug. Look at the code and explain what went wrong. Start with the observable facts and work backwards, asking why repeatedly, until you can describe the pattern that underlies the bug. Often, you should do this with a colleague, because explaining what you think happened will force you to confront your assumptions about what the program is up to.

"It blew up because subscript J was out of range."
"Why?"
"J was 10 but the top array subscript is only 9."
"Why?"
"J is a string length, and the array origin is 0, so the last character in a string of length 1 is index 0."

Look for additional surprises in the situation at the time the bug was found. Check key program variables at the time of failure, to see if you can explain their values.

"Why is the name null?"
"Why was it trying to output an error message anyway?"

Keep notes of what you did and what happened. You need to know what is really going on, and this means keeping measurements and history.

When these steps are out of the way, you are ready to ask the first question.

1. Is this mistake somewhere else also?

Look for other places in the code where the same pattern applies. Vary the pattern systematically to look for similar bugs.

"Where else do I use a length as a subscript?"
"Do all my arrays have the same origin?"
"What would happen for a zero length string?"

Try to describe the local rule which should be true in this section of the code, but which the bug disobeyed; your search for this invariant[1] will help you see other potential bugs.

"The starting offset plus the length, minus 1, is the ending subscript.. unless the length is zero."

It's more productive to fix several bugs for every one you find. Trying to describe the bugs in general terms also raises your level of understanding of what the program is doing and helps you avoid introducing more bugs as you program.

2. What next bug is hidden behind this one?

Once you figure out how to fix the bug, you need to imagine what will happen once you fix it. The statement after the failing one may have a bug in it too, but the program never got that far before: or some other code may be entered for the first time as a result of your fix. Take a look at these untried statements and look for bugs in them.

"Would this next statement work?"

While you're thinking about control flow is a good time to ask if there are other unreached parts of the program.

"Are there combinations of features I've never tested?"

It doesn't take much work to instrument a program so you can check off as you execute its various parts; and it's often surprising how much of a program doesn't work at all after the builder says it's been tested.

"Can I make all the error messages come out in test?"

Beware of making a change in one place which causes a bug somewhere else. A local change to some variable may violate the assumptions made further on in execution.

"If I just subtract one from J, the move statement later will try to move -1 characters when the string length is 0."

If you've made a lot of changes to the program already, consider carefully whether adding another local fix is the right thing to do, or whether it's time to redesign and reimplement.

3. What should I do to prevent bugs like this?

Ask how you can change your ways to make this kind of bug impossible by definition. By changing methods or tools, it's often possible to completely eliminate a whole class of failures instead of shooting the bugs down one by one.

Start by asking when the bug was introduced: when in the development life cycle could the bug have been prevented?

"The design is OK; I introduced this bug in coding."

Examine the reason for the bug in detail. Think about the process that was going on at the moment that the bug was introduced, and ask how it could be changed to prevent the bug.

"Separate data types for offset and length would have caught this error at compilation time."
"Each text item could be output with a macro which hides the subscripting calculation. Then I can figure this out just once."

Don't be satisfied with glib answers. Suppose your explanation for a bug is, "I just forgot." How can the process be changed so you don't need to remember? The language can be changed so that the detail you omitted is completely hidden, or your omission is detected and causes a compiler diagnostic. You might use a language pre-processor for this problem domain, or a clever editing tool which fills in defaults, checks for errors, or provides hints and rapid documentation. The bug may be a symptom of communication problems in the programming team, or of conflicting design assumptions which need discussion.

Consider the way the bug was found, and ask how it could have been found earlier. How could testing be made more air-tight? Could tests be generated automatically? Could in-line checking code be added that would trap errors all the time?

"I should try a zero length string in my unit tests."
"I could enable subscript checking and catch this sooner."

Systematic methods and automated tools for compilation, build, and test are always worth creating. They pay for themselves quickly, by eliminating long debugging and fact-finding sessions.

Applications of the Technique

Make a habit of asking the three questions every time you find a bug. You don't even have to wait for a bug to use the three questions.

During design and implementation review, each comment you get can be treated with the three questions. Review comments are the result of an underlying communication process which you can improve. If you feel that a reader's comment on a specification is wrong, for example, you might ask what kept your document from being understood, and how you might communicate better with the reviewer.

Design and code inspections[2] are a powerful means of finding many bugs. You can ask the three questions about each defect discovered in the inspection process. The first two questions won't turn up many new bugs if the inspection process is thorough, but the third question can help find ways to prevent future bugs.

Acknowledgments: I am grateful to the many friends and colleagues who taught me these lessons and enriched my life and work. Jay O'Dell, Rick Berman, and  Tom DeMarco contributed valuable criticism and advice on previous versions of this paper.

References

[1] Hoare, C. A. R, Proof of a Program: FIND, Comm. ACM 14, 39-45, Jan 1971.

[2] Fagan, M. E., Design and Code Inspections to Reduce Errors in Program Development, IBM Systems Journal 15(3), 1976.

[3] Saltzer, J. H.,  Repaired Security Holes in Multics, MIT CSR-RFC-5, Feb 27 1973.

Stolen / taken from http://www.multicians.org/thvv/threeq.html

Past, present, future

Hi all.

Those last months have been a dream come true. I finally moved to London as I wanted since I finish my degree and find a Java developer job in Betfair, that comparing to any other company I have known in Spain is simply awe-wait for it-some!

That's the main reason of my silence during this more than a half year. I have been struggling trying to be competent in another language, structure, methodology...

Thanks for your comments, your visits, and your time :)

For this new season, I'd like to finish my manual "Roll your own CIS", and introduce you some new projects.

1- In case it doesn't exist, I find it necessary for the world a way of avoiding avoidable testing. That's.. if you haven't modified any file, why to pass again all the tests?

I will write some more about it as I develop it.

2- One year ago, I designed some kind of new interface based on images, over the paper. I developed the first half but I was absolutely unable to complete it. I will introduce it to you and... maybe with your help, we could finish it.

3- Years ago I am trying to create a game. I'd like (along this life) to learn real navigation (not internet, over the sea I mean) and real flighting (delta wing).
Those desires try to materialize in a collaborative game of naval war. :)


See you soon

Alberto

Jenkins, the definitive guide, extract.

Today I am not giving my opinion about this book, "Jenkins: The Definitive Guide", but, if I would, I would be a good opinion, it is a definitive guide, it is quite complete and didactic. I read it through in my company today, and I want to remark the next paragraph:

"Introducing Continuous Integration into Your Organization

Continuous Integration is not an all-or-nothing affair. In fact, introducing CI into an
organization takes you on a path that progresses through several distinct phases. Each
of these phases involves incremental improvements to the technical infrastructure as
well as, perhaps more importantly, improvements in the practices and culture of the
development team itself. In the following paragraphs, I have tried to paint an approx-
imate picture of each phase.

Phase 1—No Build Server

Initially, the team has no central build server of any kind. Software is built manually
on a developer’s machine, though it may use an Ant script or similar to do so. Source
code may be stored in a central source code repository, but developers do not neces-
sarily commit their changes on a regular basis. Some time before a release is scheduled,
a developer manually integrates the changes, a process which is generally associated
with pain and suffering.

Phase 2—Nightly Builds

In this phase, the team has a build server, and automated builds are scheduled on a
regular (typically nightly) basis. This build simply compiles the code, as there are no
reliable or repeatable unit tests. Indeed, automated tests, if they are written, are not a
mandatory part of the build process, and may well not run correctly at all. However
developers now commit their changes regularly, at least at the end of every day. If a
developer commits code changes that conflict with another developer’s work, the build
server alerts the team via email the following morning. Nevertheless, the team still tends
to use the build server for information purposes only—they feel little obligation to fix
a broken build immediately, and builds may stay broken on the build server for some
time.

Phase 3—Nightly Builds and Basic Automated Tests

The team is now starting to take Continuous Integration and automated testing more
seriously. The build server is configured to kick off a build whenever new code is com-
mitted to the version control system, and team members are able to easily see what
changes in the source code triggered a particular build, and what issues these changes
address. In addition, the build script compiles the application and runs a set of auto-
mated unit and/or integration tests. In addition to email, the build server also alerts
team members of integration issues using more proactive channels such as Instant
Messaging. Broken builds are now generally fixed quickly.

6 | Chapter 1: Introducing Jenkins

Phase 4—Enter the Metrics

Automated code quality and code coverage metrics are now run to help evaluate the
quality of the code base and (to some extent, at least) the relevance and effectiveness
of the tests. The code quality build also automatically generates API documentation
for the application. All this helps teams keep the quality of the code base high, alerting
team members if good testing practices are slipping. The team has also set up a “build
radiator,” a dashboard view of the project status that is displayed on a prominent screen
visible to all team members.

Phase 5—Getting More Serious About Testing

The benefits of Continuous Integration are closely related to solid testing practices.
Now, practices like Test-Driven Development are more widely practiced, resulting in
a growing confidence in the results of the automated builds. The application is no longer
simply compiled and tested, but if the tests pass, it is automatically deployed to an
application server for more comprehensive end-to-end tests and performance tests.

Phase 6—Automated Acceptance Tests and More Automated
Deployment

Acceptance-Test Driven Development is practiced, guiding development efforts and
providing high-level reporting on the state of the project. These automated tests use
Behavior-Driven Development and Acceptance-Test Driven Development tools to act
as communication and documentation tools and documentation as much as testing
tools, publishing reports on test results in business terms that non-developers can un-
derstand. Since these high-level tests are automated at an early stage in the development
process, they also provide a clear idea of what features have been implemented, and
which remain to be done. The application is automatically deployed into test environ-
ments for testing by the QA team either as changes are committed, or on a nightly basis;
a version can be deployed (or “promoted”) to UAT and possibly production environ-
ments using a manually-triggered build when testers consider it ready. The team is also
capable of using the build server to back out a release, rolling back to a previous release,
if something goes horribly wrong.

Phase 7—Continuous Deployment

Confidence in the automated unit, integration and acceptance tests is now such that
teams can apply the automated deployment techniques developed in the previous phase
to push out new changes directly into production.

Introducing Continuous Integration into Your Organization | 7

The progression between levels here is of course somewhat approximate, and may not
always match real-world situations. For example, you may well introduce automated
web tests before integrating code quality and code coverage reporting. However, it
should give a general idea of how implementing a Continuous Integration strategy in
a real world organization generally works."

One of the best explanations about how to get into Continuous Integration in a coherent way. I hope the author (John Ferguson Smart) don't mind I took this paragraph, I am advertising the book in exchange!

Roll your own continuous integration system - Artifactory and MySQL

Roll your own Continuous Integration System (C.I.S.)

Content:

Abstract

Install Tomcat

Basic Tomcat configuration - Memory

Basic Tomcat configuration - JMX

Basic Tomcat configuration - Application Manager and permissions

Apache and uSVN

Installing Artifactory from WAR

Configure Artifactory and MySQL

Configuring Artifactory security and repositories

Configuracion de MySQL

This is a very short post, and the reason is that this is more like a remainder that the default installation cannot be used for production environments without changing default database manager (Derby) for any other. My choice is MySQL + filesystem.

Artifactory guys explained it perfectly in their wiki, just take a look, it is easy and I have just tested it, no problem.

http://wiki.jfrog.org/confluence/display/RTF/Running+Artifactory+on+MySQL

It is supposed that you have just installed it from the WAR, stop Tomcat, erase from your artifactory_home everything but "etc", it is the only folder you will need for the derby->anyother change.

Now follow the steps, if you did them alright, you will find the erased folders re-created again, and the following command will show you that some tables were created in the MySQL database: mysqlshow -u root -p artifactory .

Now continue tunning a bit this software in the next entry.

Lombok and Slf4j

I already talked  a bit about Slf4j, as a recommended interface to any implementor logger.

I have already talked about Lombok, and the fantastic way it creates java bytecode in compile time in order to make you safe some time.

Now let's reduce the time and code we need in almost every class:
private static final org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(LogExample.class);
with:
@Slf4j

That's all, put that annotation in your class, and you'll have your logger, with name "log", referring that class, and using Slf4j interface. Simply perfect ;)

If you are still a log4j lover, or maybe for other loggers, Lombok provides these annotations:

@CommonsLog
Creates private static final org.apache.commons.logging.Log log = org.apache.commons.logging.LogFactory.getLog(LogExample.class);
@Log
Creates private static final java.util.logging.Logger log = java.util.logging.Logger.getLogger(LogExample.class.getName());
@Log4j
Creates private static final org.apache.log4j.Logger log = org.apache.log4j.Logger.getLogger(LogExample.class);

More information here!

See you soon..

Choosing a Java logger, never use System.out in web applications!

Almost without notice it, loggers appeared everywhere, under that stone, behing the tree, look, another one! As an example, here we can find a handful of them (implementations, facades and factories).

Until now, we had a good reference implementation, log4j, and JUL (java.util.logging), although I always used log4j. An article talking about them. Something is clear, never use System.out in web applications, because it all goes down the drain of catalina.out, that useful file full of garbaje when other thing but server information is written.

But today we are here to talk about loggers. What should I choose? It is not as important as.. what the fuck should I do if some library uses log4j, another JUL, other MyNiceLogger, and so on...?
Two solutions for this agony, "apache commons logging" (JCL) is designed to be used as a thin bridge to the final runtime-chosen logger.

"When writing a library it is very useful to log information. However there are many logging implementations out there, and a library cannot impose the use of a particular one on the overall application that the library is a part of.

The Logging package is an ultra-thin bridge between different logging implementations. A library that uses the commons-logging API can be used with any logging implementation at runtime. Commons-logging comes with support for a number of popular logging implementations, and writing adapters for others is a reasonably simple task.

Applications (rather than libraries) may also choose to use commons-logging. While logging-implementation independence is not as important for applications as it is for libraries, using commons-logging does allow the application to change to a different logging implementation without recompiling code.

 Note that commons-logging does not attempt to initialise or terminate the underlying logging implementation that is used at runtime; that is the responsibility of the application. However many popular logging implementations do automatically initialise themselves; in this case an application may be able to avoid containing any code that is specific to the logging implementation used."

JCL has to be chosen to be effective, if a library chooses the JCL, then its logging is going to pass through your logger and you will be able to control those messages. But no solution if that library chose log4j for its messages.

Another solution, Simple Log Facade for Java (Slf4j) has the same goal, but going even further. It brings two bridges instead of one.
- You are supposed to use Slf4j, that is an interface/facade, and logger can be finally chosen at runtime.
- If your library uses log4j, a bridge called log4j-over-slf4j should be provided, it replaces log4j library, then those messages are driven through Slf4j, and finally redirected to your runtime-chosen logger again.
- An implementation should be finally included, for example, if you want your Slf4j to log into SimpleLog4j, provide slf4j-simplelog4j to drive your messages to that logger, and the runtime library simplelog4j.

Obviously, you didn't understand me.. check this out here.

Finally, by now, Slf4j is better for simplicity and flexibility, it adapts to the main log implementations, even JCL!! you don't have to change your libraries implementation whenever possible. Provide the bridges and all messages goes to your logger automatically.

Now let's dive into the final logger, now you have all your messages driven through Slf4j.
In my company, we had two main options, log4j and logback. It is funny to discover that both were created by the same person, as Slf4j was! This mess is the opus of the same unique person!!

Log4j is a good library, but Logback is better, called for replace the former, and it is said by the author of both. A list of improvements here. Mainly, it is a Slf4j native implementation, no more bridge needed, it is faster, more robust, it smells better... I knew this library, and this mess, only few weeks ago, and I have to admit my error on ignoring the "logger" mess, ignoring the libraries log and not managing it well.

My recommendation, code for Slf4j, use Logback in runtime, and use as much bridges as you need for your libraries to Slf4j. Log4j is deprecated by Logback.

See you soon...

Lombok, cleaning up your code

Much time without writing, ok, I have been working hard in company projects a bit, and the rest of the time trying to improve our tools and processes.

I want to introduce you to "Lombok", a nice tool that could bring light to the darkness of certain classes everyone has seen at least once.

The gain in this case is almost out of discussion, you annotate your code and magically it gets powers and kicks the ass to the bad guys. They are not dependencies and either aspects, because Lombok works in compilation time almost always. Indeed, inclusion of lombok.jar is not necessary in all cases, a few ones doesn't need it.


Some examples from their web:

@Getter and @Setter, it is like using Eclipse "generate getter and setter automatically" but Lombok does even easier and cleaner:

import lombok.AccessLevel;  import lombok.Getter;  import lombok.Setter;    public class GetterSetterExample {    @Getter @Setter private int age = 10;    @Setter(AccessLevel.PROTECTED) private String name;        @Override public String toString() {      return String.format("%s (age: %d)", name, age);    }  }

turns in

public class GetterSetterExample {    private int age = 10;    private String name;        @Override public String toString() {      return String.format("%s (age: %d)", name, age);    }        public int getAge() {      return age;    }        public void setAge(int age) {      this.age = age;    }        protected void setName(String name) {      this.name = name;    }  }

Those changes are made in compiled bytecode, although you can decompile this changes (delombok) and see them in Java.

Another example, let's avoid the tricky and ugly closing file even inside and exception catch.

import lombok.Cleanup;  import java.io.*;    public class CleanupExample {    public static void main(String[] args) throws IOException {      @Cleanup InputStream in = new FileInputStream(args[0]);      @Cleanup OutputStream out = new FileOutputStream(args[1]);      byte[] b = new byte[10000];      while (true) {        int r = in.read(b);        if (r == -1) break;        out.write(b, 0, r);      }    }  }

turns in

import java.io.*;    public class CleanupExample {    public static void main(String[] args) throws IOException {      InputStream in = new FileInputStream(args[0]);      try {        OutputStream out = new FileOutputStream(args[1]);        try {          byte[] b = new byte[10000];          while (true) {            int r = in.read(b);            if (r == -1) break;            out.write(b, 0, r);          }        } finally {          if (out != null) {            out.close();          }        }      } finally {        if (in != null) {          in.close();        }      }    }  }

More: automatic log declaration

import lombok.extern.slf4j.Log;    @Log  public class LogExample {        public static void main(String... args) {      log.error("Something's wrong here");    }  }    @Log(java.util.List.class)  public class LogExampleOther {        public static void main(String... args) {      log.warn("Something might be wrong here");    }  }

turns in

public class LogExample {    private static final org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(LogExample.class);        public static void main(String... args) {      log.error("Something's wrong here");    }  }    public class LogExampleOther {    private static final org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(java.util.List.class);        public static void main(String... args) {      log.warn("Something might be wrong here");    }  }

(Several implementations of log are provided).

And so on and on... check the features list

http://projectlombok.org/features/index.html

This tool is also integrated with Eclipse (through installation) and Eclipse (only by including it as a dependency). Use it wisely, but use it :)