June 2023
Imaginary Problems Are the Root of Bad Software. A great essay on why software ends up overcomplicated: developers solve the problems they find interesting rather than the ones that matter, and long chains of communication between client and developer mutate requirements beyond recognition. I can see this in myself. There's a natural pull toward the interesting, challenging problem when the simple solution is what's actually needed.
The old saying in the machine learning community is that "machine learning is really good at partially solving just about any problem." For most problems, it's relatively easy to build a model that is accurate 80–90% of the time. After that, the returns on time, money, brainpower, data etc. rapidly diminish. As a rule of thumb, you'll spend a few months getting to 80% and something between a few years and an eternity getting the last 20%.
I'm trying to build something that manages complexity. Like, you can always make your software do more. The magic is when you can make your software do more without adding complexity. Because complex things eventually collapse under their own weight.
— George Hotz, From the Latent Space podcast, around 15:50.