This is an old revision of the document!
Automation is a really important point to build reliable software. There are many tools available to do the job, and most of them are open-source.
The first tool to consider is the one that will automatically call every other ones for every modification of your source code. It is really important to have a software to do that and there are several reasons for that:
The standard steps you should do on the continuous integration tool:
While peer code review is necessary for software changes, doing part of the job with a tool help saving time and increase quality. There are several tools on the market for static source code analysis and they incredibly find many defects developers have left, sometimes because they were junior developers, sometimes because the algorithm is complex and the defect was not obvious.
One of the most important point to check when evaluating this kind of tool is the rate of false-positive. If the rate is too high, your developers will spend precious time to analyze defects that are not real problems. However, a 0 false-positive rate is probably not possible. It the tool has 0 false-positive, I can guess that it find very few number of defects. Moreover, it is important to analyze some false positive; often they are linked to complex source code that even a human will have trouble to analyze. That is why some will advice to also fix all false-positives defects: it simplifies the source code and help developers to fully understand what the program does.
I experienced Coverity on a one million sloc code base. The first time I passed it, I was really impressed of the findings. It helped improving quality already at the first run. Then, it is necessary to integrate the tool with the continuous integration system in order to maintain the quality level.
Sometimes, you will build a better development chain on an old source code base with many quality defects. In this case, it is often not possible to fix all quality issues before applying the new development chain. The good approach in this case is to consider the actual quality level as the reference, the target being to do always better but never worse.
Let's take the example of compilation warnings. Consider you have 1024 compilation warnings in the software:
Communicate this rule to your developers, and you will see the number of warnings decrease until reaching O. It can take several weeks or several month, depending of your software.