May 06, 2014
I got patches on the patches On my old blue jeans Well they used to be blue When they used to be new When they used to be clean
Everybody knows what the internet looks like, right?
Before Clown^WCloud Computing became the buzzword du jour, countless diagrams have been drawn on corporate whiteboards, in schools and universities representing the internet as a cloud. Cute. Fluffy. Non-descriptive. Just the way you'd want something to look that nobody really needs to understand but has to trust anyway.
"Ah," you say, "but in reality, the internet looks like this:"
Network connections made at peering points. A series of tubes, if you will. Another visual, perhaps more accurate and the better to represent at least some technical aspects. That's nice. Ordered. Stable. Just the way you'd want something to look that fuels such an enormous part of the world's economy.
Here's what I think the internet looks like.
It's been around for a while. It's got a few scratches, and it has been taped over countless time. But it keeps running. Doesn't look like something you'd want to get on at all. And yet you do. Every day.
Having looked a bit under the hood, I can't even give you a more reassuring report of what's going on on the inside. It's a miracle that anything works at all on the internet. It's not turtles all the way down, it's patches upon patches. It's brand-new software unleashed onto the world with minimal testing upon old, rickety libraries and frameworks that were unleashed onto the world with minimal testing and that nobody dares to manipulate and few even pretend to understand.
Almost all software out there is basically a prototype that escaped into production. It wasn't meant to be relied on, to take all this traffic, to run for so long. You don't think so? Here's an example:
There's a reason for this, and it has to do with deadlines, a willful ignorance of how software projects develop a life of their own, and a fairly good chunk of hubris on our side.
I love deadlines. I like the whooshing sound they make as they fly by.
When was the last time you delivered a project on time? The way it was planned, without compromises, with full reliable end-to-end testing, and with complete and accurate documentation? I thought so.
We suck at estimating software projects. Take your most conservative estimate, multiply it by five, and you'll still have trouble meeting that deadline, even after having applied this rule in the initial estimate. Setting a software project schedule basically means choosing an arbitrary date as the launch date, and then making compromises until you ship whatever you have at that time.
The reasons for our poor software engineering estimates are well understood -- Brooks's Law has been known for over 30 years, for example -- but we choose to ignore them.
Dunning-Kruger necessarily far outweighs impostor syndrome in the internet industry. The projects we work on are new ("You'll work on something that nobody's ever done before!"), but we believe that similar past experiences will make this project a walk in the park. We're smart, we've done this before. Well, not that exact same thing, but something similar. It had to do with computers, anyway, so this shouldn't be a problem.
And in the highly unlikely event that the software we write is not perfect (gasp!), we have no doubt that changing it later will be easy anyway. After all, software is infinitely malleable. We can rebuild it. We have the technology. But we don't have to think about that right now, we're on a schedule.
Unfortunately, changing software after it has been released becomes increasingly more difficult. And laborsome. We'd have to retain backwards compatibility -- often times for bugs others have come to rely on -- or decide to break other software, clients, or consumer expectations. This gets tedious quickly. You know what's much more exciting? That new project over there, where all the smartest engineers in the company are working on this ground breaking new product for make benefit glorious ego of self. Shiny!
In the mean time, somewhere on the internet, a software engineer encounters our product -- bugs, inconsistencies, warts and all. Scoffs and snickers: "Why didn't they just (use NodeJS, YAML, use 64bit ints, write this whole thing the way I would have)?" So yeah, they go and write their own. It'll be so much better than this piece of crap! Witness the birth of another web server, itself further evidence of Sturgeon's law.
Somewhere else an SRE, or perhaps a System Administrator, accepts our products' problems and works around them, because that's what we do when we can't write our own funamjing. They probably used a shell script wrapper around a perl script, or perhaps a custom, self-extracting zip archive "binary" of some python code that can only be built on this one guy's laptop.
And that is how the internet is held together. Dirty Deeds Done Dirt Cheap.
May 06, 2014