Can humans make AI any better? Another great Welch Labs video about the bitter lesson, contextualizing the recent Dwarkesh interview with Rich Sutton.
Many people misunderstand the bitter lesson. It's not that some solutions are "past" it while others aren't. It's a ladder of increasingly general approaches to the same problem, with a persistent tradeoff between heuristics that perform well now versus general methods that win in the long term, given enough compute.
In my experience, only three things matter for using coding agents effectively:
- Clarity of mind about what you want.
- The ability to put it into words precisely.
- A system to verify the outcome and improve.
The first two are harder than they sound. If you don't know what you want, that's fine, but be conscious of it. Use the agent to gain clarity, don't just let it run when you're still figuring things out.
For the third, give your agent a way to check its own work. A robust feedback loop with type checkers and robust tests turns guessing into learning.
Model choice, prompting tricks, and context management are far less important.
None of this is specific to AI. These are the same skills that make people effective in general: clarity, communication, and the ability to learn from mistakes.
LLMs had a rough start to 2026.
Last week, our AI agent told a user their Monday meeting was on Sunday. The reason? In 2025, January 12th was a Sunday—and that's what the model learned during training.
We're seeing this pattern across many date-related tasks: models confidently inferring the wrong day of the week, or hallucinating years that weren't in the input text.
The obvious fix is to inject today's date into every prompt. However, that's not always what you want. For example, if you're extracting dates from text, adding context about "today" can bias the output and cause hallucinations of its own.
Is this the new Y2K bug we'll have to deal with every January? Or will future models be more robust to year transitions?
One of the hardest adjustments to AI coding tools has nothing to do with prompting. It's overcoming your attachment to code.
When you write code by hand, you accumulate emotional investment with every line. Throwing it away feels like a waste. So you keep patching, extending, and defending code in reviews that should have been discarded.
Use AI to break this pattern and don't fall into the attachment trap. Code becomes cheap enough to be treated as exploration rather than construction. Prototype quickly, build your mental model, and find the simplest solution that solves your problem. Then start fresh with clarity.
The developers who get the best results aren't the fastest prompters. They're the ones who know when to start over, and when to walk away entirely.
The most important work skill in today's age of AI is, ironically, being generative.
AI removed the execution bottleneck. So the new constraint is how quickly you can generate ideas worth building—and whether you actually build them or get stuck in possibilities. I first heard this advice an episode of Lenny's podcast with Replit's founder Amjad Masad in November 2014.
Following this advice has served me well in 2025. I stopped reading about Claude Code and tested it myself, then convinced our engineering team to adopt it. When I saw an opportunity for an AI email assistant, we built Ren Agent instead of strategizing about it.
So if you need a resolution for 2026: AI is generative. Make sure you are too.
Looking for a last-minute gift for someone technical who wants to get into AI, or an entry point for yourself?
I highly recommend "Natural Language Processing with Transformers" by Lewis Tunstall, Leandro von Werra & Thomas Wolf. It's a practical guide to training and scaling transformer models, written by the people who built the library at Hugging Face.
This is where I learned the fundamentals of modern transformer models a few years ago, and despite being from 2022, the notes I took while reading the book remain the resource I go back to most on the topic.
Happy Holidays!
This new screen aims to solve a rather obvious problem with all of the AI apps: what do you use them for? All of the options on this screen are achievable through a chat interface, but you need to know what to ask for, which is actually step 2 of the process: first you have to know what is possible, and most people don't. This screen aims to solve that: there are obvious filters you can use, and ideas for images you might want to create, like a Christmas card. Again, all of these are doable from a text interface, but there is a reason why purely text interfaces are the domain of so-called graybeards: it's not the typing that is the problem, or even knowing what to type: it's knowing what you could type.
[...]
To summarize, one role of product is to show you what you can do; another role is to inspire you to come up with more of your own ideas.
— Ben Thompson, Stratechery: ChatGPT Image 1.5; Apple v. Epic, Continued; Holiday Schedule
Last weekend, my son was working on a Bandolino puzzle where he had to match questions to answers with a piece of string.
OpenAI just released GPT–5.2 last week, claiming it "performs at or above human expert level" across vision, math, and physics benchmarks [1]. I was curious how long it would take GPT–5.2 to solve it—it failed completely.
How can such a powerful model fail at a task that a four-year-old solved in under a minute after seeing it for the first time?
Andrej Karpathy coined the term "jagged intelligence" for this phenomenon [2]. LLMs can solve complex problems that seem hard to humans while failing at tasks that seem trivially easy. Unlike human intelligence, where abilities tend to correlate and develop together, LLM capabilities are jagged and unpredictable.
To put it another way: while these models are extremely powerful, they can't be trusted.
What does this imply for deploying LLMs in production settings?
(1) Benchmarks give you no guarantees—you have to evaluate models on your own tasks.
(2) Your overall system has to be tolerant of these jagged edges. Use LLMs for the tasks that they are good at and keep a human in the loop on all critical decisions.
(3) You have to take security extremely seriously. Meta's "Agents Rule of Two" is a great framework for AI agent security that is simple to remember and apply in practice [3].
At Ren Systems, we leverage LLMs extensively to create value for our users. But it's always our users who ultimately take action.
What's one lesson you've learned deploying LLMs in production?
(References/links in first comment below)
References/links
[1] Introducing GPT-5.2 – https://openai.com/index/introducing-gpt-5-2/
[2] Andrej Karpathy on jagged intelligence – https://x.com/karpathy/status/1816531576228053133
[3] Meta's Agents Rule of Two – https://ai.meta.com/blog/practical-ai-agent-security/
Last week, I posted about AI coding tools and engineering productivity. It got more attention than expected and sparked a debate in comments I didn't anticipate: Is the merge request rate a good measure of productivity?
Fair question. Here's my take:
I used the rate of merged MRs as a proxy metric for productivity. Like all proxies, it's imperfect, but I think it's better than most alternatives, at least in our case.
If you can break a complex feature or refactoring into multiple small, independent MRs, that's not gaming a metric. That's good engineering. Your reviewers will thank you, and your users will benefit. More power to you!
If you're gaming a metric, or worried that others are, the issue isn't the metric. It's your company culture.
So yes, for our company at our current stage, with our team and culture, I believe the rate of merged MRs is a useful signal for engineering velocity.
For what it's worth: at Ren Systems, we don't measure engineers by MR rate or any similar metric. Neither individually, nor collectively. I shared this observation as a quantifiable measure of the acceleration we feel as a team in our daily work.
But this debate is secondary to my main point: I wasn't claiming AI made us magically faster. The observation was that our merge rate increased. The argument was that strong foundations are the reason we're accelerating.
Are AI tools contributing? Probably. But they're not the cause. They're the amplifier.
Since our engineering team adopted Anthropic's Claude Code in June, we're shipping code to production over 50% faster.
Do AI coding tools make developers more productive or is it all hype? I think that's the wrong question. The real question is: do you have the foundation to use them well?
Teams with solid architecture, test harnesses, and reliable CI/CD are accelerating. Teams without those fundamentals are drowning in code they can't trust.
At Ren Systems, our engineering team spent years building those foundations, thanks to the hard work of people like Jan Giacomelli and Giorgio Nicoli. Now that investment is compounding.
Have AI coding tools made us 10x more productive? No. But we're shipping more, with confidence, and nobody's losing sleep. Features that used to take months now ship in days. Our Salesforce integration, on our sales team's wishlist for years, took one week to build.
AI coding tools don't replace good engineering. They amplify whatever's already there.
I'm trying to build something that manages complexity. Like, you can always make your software do more. The magic is when you can make your software do more without adding complexity. Because complex things eventually collapse under their own weight.
— George Hotz, From the Latent Space podcast, around 15:50.
The old saying in the machine learning community is that "machine learning is really good at partially solving just about any problem." For most problems, it's relatively easy to build a model that is accurate 80–90% of the time. After that, the returns on time, money, brainpower, data etc. rapidly diminish. As a rule of thumb, you'll spend a few months getting to 80% and something between a few years and an eternity getting the last 20%.
What the large language models are good at is saying what an answer should sound like, which is different from what an answer should be.