User Tools

Site Tools


a_development_chain_to_build_reliable_software

This is an old revision of the document!


A development chain to build reliable software

Automation is a really important point to build reliable software. There are many tools available to do the job, and most of them are open-source.

Continuous Integration with Jenkins

Website: https://jenkins-ci.org/

The first tool to consider is the one that will automatically call every other ones for every modification of your source code. It is really important to have a software to do that and there are several reasons for that:

  • It does boring and repetitive tasks
  • It does always the same things, whereas a developer may sometimes make mistakes
  • It can build your software 24 hours a day
  • It can build a software for each modification of your source code

The standard steps you should do on the continuous integration tool:

  • Detect modification on the source code repository (svn, git, …)
  • Build the software (gcc, Microsoft Visual Studio, …)
    • Use complier features to avoid potential issues, for example 0 warnings is a best practice.
  • Run unit tests
  • Run automatic integration tests
  • Do some custom checks on the source code
    • Use custom scripts to check coding guidelines
    • Use tools to do static source code analysis (Coverity, …)
  • Build release packages

Static Source Code Analysis with Coverity

Website: http://www.coverity.com/

While peer code review is necessary for software changes, doing part of the job with a tool help saving time and increase quality. There are several tools on the market for static source code analysis and they incredibly find many defects developers have left, sometimes because they were junior developers, sometimes because the algorithm is complex and the defect was not obvious.

One of the most important point to check when evaluating this kind of tool is the rate of false-positive. If the rate is too high, your developers will spend precious time to analyze defects that are not real problems. However, a 0 false-positive rate is probably not possible. It the tool produces 0 false-positive, I can guess that it also finds very few number of defects. Moreover, it is important to analyze false-positives; often they are linked to complex source code that even a human will have trouble to analyze. That is why some will advice to also fix all false-positive defects: it simplifies the source code and help developers to fully understand what the program does.

It is important to notice that static source code analysis can help finding bugs in your software, but also errors that are not yet bugs but that can become bugs after a new modification.

I experienced Coverity on a one million sloc code base. The first time I ran it, I was really impressed of the findings. It helped improving quality already at the first run. Then, it is necessary to integrate the tool with the continuous integration system in order to maintain the quality level.

Efficient Code Review with ReviewBoard

Improving the Quality of legacy source code

Sometimes, you will build a better development chain on an old source code base with many quality defects. In this case, it is often not possible to fix all quality issues before applying the new development chain. The good approach in this case is to consider the actual quality level as the reference, the target being to do always better but never worse.

Let's take the example of compilation warnings. Consider you have 1024 compilation warnings in the software:

  1. In the script that checks the number of warnings, consider the build fails if at least one new warning has been introduced. If the warnings count is still 1024, do not break the build.
  2. If for the current build, there are less warnings, for example 1020, consider this as the new reference
  3. Return to step 1 while there are remaining warnings
  4. When there are no remaining warnings, modify your compilation flags to consider warnings as errors

Communicate this rule to your developers, and you will see the number of warnings decrease until reaching O. It can take several weeks or several month, depending of your software.

Resources

  • Mars Code: a very good conference by Gerard Holzmann from JPL at NASA. It deals with the software used to send Opportunity to Mars.
a_development_chain_to_build_reliable_software.1450286525.txt.gz · Last modified: 2015/12/16 18:22 by sgripon