Building Software is a Losing Game

If you’ve ever played an RPG video game – particularly a Japanese RPG like Final Fantasy or Dragon Quest – you might have encountered what is known in the genre as a Hopeless Boss Battle. This is a mechanic typically used for the sake of the plot, in which our heroes have to face an impossibly powerful enemy in a battle they’re supposed to lose.

Building software is very much like one of those battles, except that our enemy is not a level 9000 dark knight or a Kraken-like monster with an impenetrable shield. Our battle is against a much scarier beast: complexity.

Complexity is the single major problem in software development. It is what makes systems hard to understand, maintain, and modify, and it’s the main cause of software products being delivered late or not at all. In their classic paper Out of the Tar Pit, authors Ben Moseley and Peter Marks rightfully call the enemy by its name:

Complexity is the root cause of the vast majority of problems with software today. Unreliability, late delivery, lack of security — often even poor performance in large-scale systems can all be seen as deriving ultimately from unmanageable complexity.

There was a time when I thought I could defeat complexity. I felt that if I just had enough time and resources, I could design and implement a perfect version of my software that would be completely free of bugs and edge cases. I thought that if I could just refactor all the legacy code and handle every last piece of tech debt, I would finally be free of the curse of complexity, and my productivity would be through the roof!

But I was wrong. As most seasoned developers know very well, perfect codebases don’t exist because complexity always finds a way to creep into our projects. There are always tradeoffs that we have to make, changes that we need to adapt to, and real, complex problems that we need to solve. Even the simplest design we could possibly think of will eventually fail to keep complexity out of our codebases.

We’ve already lost

So the first step in the battle against complexity is to admit defeat. If that sounds depressing is because, well, it is. But it is also liberating—if the perfect design doesn’t exist, then there’s no pressure to create it.

It’s a hopeless battle because any piece of useful software comes with a certain amount of essential complexity attached to it, which is the complexity of the problems it solves and the domain it lives in. This type of complexity is the entire reason the software exists in the first place, so there is really no way around it.

We do have some level of control over the accidental complexity of our applications, however, which is the complexity of the tools and patterns we use to build them. But while this type of complexity is more manageable, it is also inevitable. As time, frequency of changes, and the number of contributors to the codebase increase, so does its accidental complexity. We can manage and encapsulate it, but we can never completely eliminate it.

Simplicity is Hard

Complexity in a codebase builds slowly over time with every line of code we write and every tiny decision we make. In the early stages of a project, these tiny bits of complexity might seem harmless. A little code smell here, a small broken window there, a few “I know this is a hack but I don’t have time to fix it right now” here and there. But complexity is exponential—these tiny bits add up, and before we know it, our greenfield project has turned into an unmaintainable piece of legacy code seemingly overnight.

But it doesn’t always have to be that way. Our job as software engineers is not really to defeat complexity, but to manage it—to put the beast to sleep as long as we possibly can, so that we can build software that would last longer.

My favorite book on this subject is A Philosophy of Software Design by Professor John Ousterhout. The book is not only packed with clear, practical advice for dealing with complexity, but it also states the importance of our role as software designers right at the beginning of chapter 1:

If we want to make it easier to write software, so that we can build more powerful systems more cheaply, we must find ways to make software simpler. Complexity will still increase over time, in spite of our best efforts, but simpler designs allow us to build larger and more powerful systems before complexity becomes overwhelming.

The fight against complexity might be an impossible one, but it is one we must fight anyways. If complexity is the most fundamental issue in software development, then managing it must be the most important part of our jobs.

Of course, simplicity is not easy. Coming up with a simple design is a lot harder than building a complex one, and it only gets harder once we start implementing and making changes to it. But just like the heroes of a Role-Playing Game facing an unwinnable battle, we can’t just give up. If we want a chance at building great software, we must rise to the challenge, look directly into the eyes of complexity, and fight it with all our might.