[SC-L] "Checklist Manifesto" applicability to software security

[John Wilander]

This is a question of great interest to me. Thanks for the links Jeremy and
Brian.

Atul Gawande points it out nicely in the interview: "The problem [for you as
an expert] is that the volume of knowledge exceeds the capacity of your
brain." When I give talks or training on software security I always include
a section on complexity. I'll give you the key points I usually cover ...


*** A. Complexity in software has five dimensions ***

1. Scale. More code means higher complexity. This is often hard to avoid
although code refactoring can reduce the code mass somewhat.

2. Diversity. The number of technologies, frameworks, languages, versions,
and even varying coding conventions -- they all drive complexity. This can
be handled with policies but developers often oppose a strict list of what
to use and what not.

3. Connectivity. On code level it's about coupling and cohesion. On a
systems level it's about architecture. Even higher it's about
interconnecting systems and services, perhaps cross-organization. In the
server room it boils down to cables. High connectivity means high
complexity. This too can be handled, but lack of communication between
people is often an obstacle. I've been to customers who were convinced for
years that their test system and production system were completely
separated. Then they changed outsourcing partner and discovered the truth.

4. Dynamics. Could the system even be described in a state diagram? On any
reasonable level? As the number of states the system can be grows, so does
its complexity. I'm not sure if we have the tools or skills to manage
software dynamics. Yet.

5. Refinement. Over time software get refined. If you go to a large bank and
check out their core systems you'll find years and years of refinement.
Understanding why things are done the way they are in a highly refined
product is complex. This kind of complexity can be lowered through
documentation, but not avoided.

I've attached the picture I use to explain these dimensions. Not a product
of art but it works :).


*** B. Be aware of the Complexity Barrier ***

The testing guru Boris Beizer has founded two interesting laws regarding
software bugs, testing, and complexity:

First law: The pesticide paradox. Every method you use to prevent or find
bugs leaves a residue of subtler bugs against which those methods are
ineffective.

Second law: The complexity barrier. Software complexity (and therefore that
of bugs) grows to the limits of our ability to manage that complexity.

I find the complexity barrier to be very interesting. We're not trying to
build the ultimate Notepad.exe over and over again (someone should though).
Instead we're diving headfirst into Rich Internet Applications as soon as
it's possible. That's human nature.


*** C. We need computers to analyze computers ***

The complexity of the systems we build is so big that we nowadays need
computers and software to understand them. This is where checklists come in.
We do use checklists all the time in the form of static analysis, both in
compilers and in tools. That's where many of our (low-level) checklists
belong. But we have to get developers to use them.


   Regards, John Wilander
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://krvw.com/pipermail/sc-l/attachments/20100107/238f8b1d/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: john_wilander_complexity_in_5_dimensions.png
Type: image/png
Size: 210022 bytes
Desc: not available
URL: <http://krvw.com/pipermail/sc-l/attachments/20100107/238f8b1d/attachment-0001.png>