Doing Complex Things Is Complicated
But like, why tho?
Doing complex work is complicated. Intuitively, this is something that we all know. This is not a controversial statement. We can spend our entire careers, devote our entire vocation, to answering the question of “How do we do the complicated thing well?” It takes a lot of effort to even articulate what we mean by that question, for any given domain, let alone developing a useful answer we can apply in the future. Luckily for us, many very smart people have done this and written books about what they found.
Bent Flyvbjerg studies how megaprojects – across a range of industries, including software – succeed or fail to come in on time, on budget, and deliver the promised value. He notes that only a small, small fraction of any project delivers all three, and looks to understand what makes these successful projects different from projects that blow past timelines and budgets only to underdeliver on their promises.
Christopher Alexander has a different measure of success – one that’s far more qualitative that Flyvbjergs. Alexander asks what makes the built environment feel alive, and enrich the lives of it’s inhabitants. Some spaces are qualitatively fantastic, many more qualitatively terrible. How can we make more of the former, and repair more of the latter?
Both Alexander and Flyvbjerg note that big, complex projects are hard. Flyvbjerg has spent decades collecting project data around costs and timelines, and finds that only 8.5% of big projects come in on budget and on time. Of those, the majority fail to deliver on their intended benefits. Only 0.5% of all projects manage to be on budget, on time, and deliver what they intend.
99.5 perfect of projects go over budget, over schedule, under benefits, or some combination of these. Doing what you said you would do should be routine, or at least common. But it almost never happens. • How Big Things Get Done, Flyvbjerg 2023.
Not only this, but the overruns tend to be catastrophically bad. Flyvbjerg observes that big projects follow a fat-tailed distribution, that is a non-normal distribution where extreme outcomes are far more likely than we might expect. For IT projects, for example, roughly 1 in 5 projects are in the tail end of the distribution, and the mean cost overrun of those projects is 447% – on par with cost overruns related to nuclear waste storage
Alexander explains why this is an anticipated, even expected outcome as early as 1964 in his book Notes on the Synthesis of Form. He notes that in the process of designing and creating a structure, there are an enormous number of questions that we must answer. These can be as simple as “how big should this window be” or “what should we name this function” or as large as “how many square feet is the building” and “what is our underlying data model.” The answer to each one of these questions can create impacts on any number of other questions. Alexander noted that this creates a graph network, nodes of design questions linked by edges of consequent impact. Around the same time, Kunz and Rittel developed a similar formalization of describing complex problems in their IBIS framework, resulting in a similar graph topology of issues, positions on those issues, and arguments in support or opposition of those arguments.
The idea for Alexander and the purpose of IBIS was to decompose the large graph into smaller pieces that are more easily understood and addressed, minimizing the ripple effect of consequent impacts. Basically, what’s the most efficient way to work through the nodes in the graph? As it turns out, this question was mathematically shown to be NP-Hard by Richard Karl in 1972 – in short, you’re not gonna get it right. The best we can hope for it “not terrible”, and Flyvbjerg shows us what “terrible” can mean.
We can see what’s happening in the disastrous projects from this perspective. Flyvbjerg notes that one of the most common process characteristics of all these failed projects is the tendency for those in charge to “think fast, act slow”. Leaders rush to commit to and start a project for a variety of reasons, make a fast and loose plan about what it will take to accomplish, and assume they’ll figure out the rest as they go. To explain this process from Alexander’s perspective, leaders commit to solving a graph network of problems before they know the complete size of that network, let alone its topology and how to deconstruct it. They then enter that graph at basically random – the questions that they can see clearly and seem like a good enough place to start – then blindly walk that graph, discovering the topology as they bump into it. Problems cascade, decisions made early have unforeseen consequences, and timelines get pushed, which means more money is needed. Ultimately, we start cutting off chunks of the graph as “won’t do, out of scope” in order stem the bleeding which means we commit again to not delivering and unknowable amount of value. Mathematically, the sheer number of bad ways to go about the project out number the good ways by such a huge margin that it’s surprising that any projects are successful.
A story collected in the Failure Whitechapel: Documents of Contemporary Arts illustrated the problem more poetically. A captain must cast off for the open ocean at dusk, through a storm. Visibility is almost zero, but the captain has an idea – an image – in their head of the shape of the bay, the location of the rocks, the route the ship must take. At the point, there are two possibilities: the first, the captain successfully navigates their ship out to sea; the second, the captain runs aground against unforeseen rocks and drowns. In only one of these scenarios does the captain learn anything about the reality of the world around them.