Fooled by Randomness

Is software development a science or engineering discipline in which we can strive for perfection? Are perfectly executed software plans possible in today’s market? I believe that software development, in general, is fraught with uncertainty and is therefore governed, at least in part, by random events. Assuming for an instant that this statement is true, then there are tremendous implications on how software projects are developed and managed. The Agile movement is largely based on this concept, yet it’s not a slam-dunk for many people.

In the pop-finance book “Fooled by Randomness“, the author N. N. Taleb makes a case that most, if not the whole, of the financial markets industry is governed by chance; randomness, not the skill of individual market traders, accounts for the failures and successes of the numerous funds and similar financial vehicles. Taleb argues that our population is generally uneducated and inherently unskilled at dealing with the statistical reality of situations fraught with randomness (like financial markets), and that certain biases (like the survivorship bias) skew our thinking to confuse “success” with “skill”. He argues that it is actually luck that determines success in financial markets, not the skill of individual traders. Taleb contrasts the financial markets industry (purely “luck” based) with other industries that rely less on luck and more on skills; he uses the example of dentistry as a profession that relies largely on the skill of individual dentists as opposed to how lucky they are in determining success.

Where is software development on this luck-skill spectrum? I believe software development lies somewhere in between financial markets and dentistry on the spectrum of luck vs. skill based industries. Certainly, the skill of the people involved has an impact on the outcome of software development projects. However, I do believe that software development is fraught with uncertainty and therefore relies to some extent on good luck. Here are just a few areas that I’ve observed to cause uncertainty in software development projects:

  • Software is not a physical science. Unlike bridge building, there are no physical laws that underpin software development. As a result defects are common and frequent. And further, we build systems on top of platforms that were themselves built on shaky foundations.
  • Changing product goals due to business climate. Software is typically built in response to business needs, not the other way around!
  • Interpersonal interactions. There are many people and many personalities involved and the interaction of complex systems (like people) can rarely be predicted.
  • Personnel changes. People come and go, and software knowledge workers are not “fungible resources” (like it or not!).
  • Software is hard to visualize until there is some software to sample. “I’ll Know It When I See It” is a common refrain.

This list could go on for a while, but hopefully you get the general idea. For a more complete list, see Craig Larman’s excellent book “Agile and Iterative Development: A Manager’s Guide.”

(Important Aside) Note in the above paragraph that I used the word “uncertainty” and not “risk”, as there is a big difference: risks can be mitigated while uncertainty cannot regardless of the measures we take. I’ve heard many software professionals talk about risk mitigation as a driving force in software methods. Unfortunately, risks are only one side of the coin, the other is even nastier: complete uncertainty that cannot be resolved regardless of the time spent (think BDUF) trying to mitigate.

So it appears that software development is more a blend of art and science rather than a strict science that can be calculated with mathematical certainty. While this may be a hard pill to swallow for some, it certainly makes sense that Agile methods like Scrum and XP are better equipped to juggle the complexity of software development than traditional development methodologies. Traditional methodologies take a formulaic approach to development, featuring libraries of cookie-cutter prescriptions to account for all or most possible scenarios. Of course this won’t work very well with a nearly limitless set of possibilities. It is best to set high level goals and work in small steps toward those goals, making adjustments for the reality of the situation along the way.

If this is true, then there must be a collective realization that software development projects cannot be controlled like other “engineering” projects. We’re not building a bridge, or even shooting a rocket into space; we’re doing something harder. The iron triangle variables of cost, schedule, and requirements cannot be fixed in advanced with any expectation of project success. But the Agile folks have been saying this for a while yet many people out there still hang on to the belief that software can be “calculated” by applying more rigor to methodology. To those people, I suggest reading Taleb’s book even if you are not interested in financial markets because it highlights a common flaw in our collective psychology, the fear of the uncertain to the point of denial. Our instincts (in general) are to cling to anything that promises predictability and repeatability. But if we are to succeed in disciplines that are affected by randomness to a large degree, software development among those, we must try to fight this instinct and open our minds to dealing logically with uncertainty.

Victor Szalvay

Victor Szalvay currently leads product development for CollabNet’s ScrumWorks® product suite. In that capacity, he works closely with customers, stakeholders, and the development teams to deliver high business value each release cycle. With more than 150,000 active users worldwide, ScrumWorks is used by more than half of the Fortune 100 and boasts the largest market share of any Agile management tool.

Posted in Agile
2 comments on “Fooled by Randomness
  1. ontwikkeling projecten says:

    I think software development is uncertain because of the speed we live with and everything is changing every day and new things pop up and that’s way is hard to be perfect in anything

  2. nicoduca says:

    Software engineers suffer from excessive introspection and self-doubt about the existence of the field. This paper takes a look at the history of engineering to find examples consistent with the practice of software engineering. We find that software engineering does not have to wait until there is a mature underlying science, as engineering practice has generally led science. That software engineering has characteristics of design practice that closely match what is normally considered to be engineering design. And that software engineering has many attributes of any other new branch of engineering. Finally we consider the reasons why software engineering is not recognized, mostly due to inconsistencies in practice.

Leave a Reply

Your email address will not be published. Required fields are marked *

*