Traditional vs. Non-blocking Continuous Integration Systems

Unlike traditional CI Systems which only perform a passive monitoring of the quality level of software project branches, non-blocking CI systems are actively involved in the pre-commit qualification process in such manner as to ensure that no regressions occurs in a pre-defined set of QA verification criteria.

The end goal of every software project is to reach a certain quality level by its release date, meaning it should be free of any software defects causing failures in the respective QA verifications. While some of the defects may be day one defects (like missing or incomplete features, for example), others may be regressions caused by the software changes being integrated during the project's lifetime. Some of these regressions can be extremely damaging, blocking the project's development activities. The method by which these regressions are addressed is what distinguishes projects using the non-blocking CI systems from those using traditional CI systems.

Traditional CI systems periodically perform QA verifications in order to detect regressions in the integration branch. Unfortunately this means these regressions are already impacting the project by the time they are detected, they need to be repaired for the project to make progress.
But repairing regressions often require human intervention, most traditional CI system have little or no automated support whatsoever for such activities. The primary reason for that is the lack of a complete, deterministic algorithm that would always be capable of producing a repair solution (other than the project moving backwards to the last stable label, of course).

Typically repairing a regression consists of identifying the committed software change which caused it and backing it out. But neither identifying the culprit nor backing it out are always simple, fast enough and/or even possible, which makes projects based on traditional CI systems susceptible to congestion, a significant limiting factor when it comes to the project's speed and scalability.

In an attempt to prevent regressions many projects make mandatory pre-commit QA verifications of the software changes. But, especially for large scale projects, the coverage of these pre-commit verifications is often just too small compared to the desired project quality level due to the limited verification costs. Checking that the changes don't break the build offer no protection whatsoever against the built image(s) being DOA and failing all feature testing, for example. Also, even for the covered functionality, individual, uncoordinated pre-commit verifications can be ineffective in preventing regressions. But they are effective in reducing the regression occurring rates.

Non-blocking CI systems rely on coordinated pre-commit QA verifications to identify and prevent faulty software changes from being committed into the integration branch and thus from causing regressions. No repairs needed. And, more importantly, it is a process which can be entirely automated. The coordination ensures that no pre-commit QA verifications is started until the previous such verification is completed and, if successful, the verified changes are committed to the integration branch.
In other words the pre-commit verifications of the changesets being committed are completely serialized - the only way of actually preventing regressions. Yes, serialized. And no, that doesn't necessarily mean they compromise on scalability. In fact our patent-pending ApartCI product was specifically designed to outperform traditional CI systems, especially on very large scale projects.

The following table summarizes the significant differences between software development projects using traditional CI systems and those using non-blocking CI systems.