bitlog.com

Jake Voytko's personal log of bits

Blog posts

How software engineering at Hinge has changed

This was written as part of the Pragmatic Engineer essay contest.

Software engineering has greatly evolved at Hinge over the past few years thanks to recent advances in agentic coding. Every month we incorporate new tools and techniques. We’re accomplishing engineering feats on timeframes that were impossible when I joined Hinge at the beginning of 2023.

The best part is that none of our techniques are complex or complicated. There is no magic. Everything is actionable and repeatable. If you work as a professional software engineer at an app or web company, you can likely adopt most of them. The only barriers are LLM budget, time, and a willingness to own the outcome. Honestly, many of you have probably adopted these, and you probably have your own strategies that you could share with us to make us move even faster. I’d love to hear them!

But first, let’s back up. Hinge is a dating app, and dating apps need humans in the loop. There are many reasons we can’t just hand Hinge over to the machines and go to lunch.

The first reason is taste. Imagine that you’re a software engineer working on tools for software engineers. It wouldn’t surprise me if you built and shipped a small feature within 30 minutes. If the problem were small and the solution were easy to verify, why would you need more time than that? But now imagine that you want to make the app resonate specifically for Gen Z users while still serving all of your existing users. That’s much harder, especially if you are in another demographic. Is there any human on this earth that can “build for themselves” across every national, political, gender, sexuality, and generational divide you can name? Absolutely not. You must work in aggregates. You need to perform a bunch of research. A lot of the research can be collected via surveys, but ultimately, you also need to find ways to interact directly with your audience and incorporate those ways into your process. Skilled designers and product leaders are needed to synthesize this information and propose new features. And you need that human touch and the taste that nails the vibe while straddling all of those lines.

Also, safety is critical to the experience on dating apps. Users tell us things that they might be afraid to tell their own family or declare publicly. So we must remain in control of our app’s experience because the stakes are too high. Ultimately, we recognize that we have ownership of what gets shipped into production, and we act accordingly.

It’s important to contextualize all of our LLM wins: they are done in an environment where there is real human impact, and human judgment still reigns supreme. And the results are really exciting. We have still found massive gains in four areas: prototyping, on-call incident response, tooling, and engineering. Let’s dive into each of these areas and go over specific techniques that have accelerated us.

Prototyping is easier than ever

A year ago, I joined two other recs engineers (and a talented MLE and one plucky user researcher) and together we built a recommender in four days for a hackathon. We vibecoded as hard as we could, we tuned it as hard as we could. Fueled by expertise and memes, we got our prototype over the finish line for second place. It was the seedling for a project that eventually hit production.

Earlier this year, we had another hackathon. I picked another recs project that had comparable backend difficulty to the previous year’s. But there was something different: I teamed up with two talented mobile engineers and I wrote the entire prototype recommender system by myself in three days while the mobile engineers built a custom UI to show it within the app. Good enough for third place, and it also was the seedling for a project that eventually hit production. Yes, this was a step backwards for podium finish, but it was a leap forward in that agents help us accomplish things on timelines that used to be out of reach.

Did I become a superhuman prototyper within a year? No way. The LLMs got better at generating prototypes. And when it’s easy for engineers to generate prototypes, engineers can get more product features into production. The old days are gone. You don’t need to craft an argument that’s so compelling that every person in the company bends their priorities out of the way so that your idea has room to be prioritized. You can just build something and show them, “hey, I think we should do this.” And if it’s good, people will pay attention.

This is happening at Hinge right now. I have done it. Others have done it. You can do it. If you have conviction that something should be built, fire off a prototype while you work.

You can make this an intentional part of your process. At Hinge, we’ve had success with official small teams with a few disciplines. At the beginning of the year, we were doing experimental machine learning model work, but we weren’t sure if we were on the right track. So we created our own Canterbury Tales cast of a Backend Engineer, a Machine Learning Engineer, a User Experience Researcher, and a Product Manager, and we rapidly iterated on prototypes with internal testers until we landed on a successful formula.

This approach has also succeeded outside of my domain of backend recommendations. Another team recently had a lot of success teaming up a Product Manager, Mobile Engineer, and Designer, and they faked the entire backend from the client as they continued to work on a UX-heavy experience.

There’s no right way to do it. Fake the backend for internal prototypes if you’re a frontend engineer. Fake the frontend if you’re a backend engineer. Just get something working and show people.

What happens after the prototype phase? Once the prototypes hit a critical mass of acceptance, they transitioned into more normal projects: Figma designs, Trust and Safety reviews, design docs, implementation sprints, etc. The prototypes are just that: prototypes. We borrow what we can from the prototypes and ensure that the shipped production system is implemented at a high quality.

Bots are helping out with on-call

Here’s a hypothetical incident: You hear a loud noise. Where are you? You’re in bed. Maybe it’ll stop if you don’t move. It’s still going. Then you find your phone and stare at it for a second. You’re being paged. It’s three in the morning. Now you remember all of those responsibilities you have. Right now, the most important one is that you’re on-call. You ACK the page and sit up in bed. You need to find your laptop. You go get it. You log on and go into Slack.

And then you click into @oncall-bot’s thread in the incident channel.

Here, you find a fully-automated attempt at an investigation. A critical service has an elevated gRPC failure rate. @oncall-bot notes that the service’s CPU is much higher than normal and the service hit its autoscaling limit and can no longer spin up new pods. It investigates a bit further. There are new stack traces coming from that endpoint. @oncall-bot examines the stack trace. An experimental feature was recently added in the code paths, and its code branches are taking too long for some users. @oncall-bot can’t figure out why it’s taking so long, though. You ask @oncall-bot to make a pull request to turn the experimental feature off while you double-check everything that it said. It all looks correct. You get an approval from another on-call engineer, you turn the feature off, you make sure the graphs normalize, and you go back to sleep. This is tomorrow’s problem.

We’re only at the beginning of our on-call automation journey, but the results are encouraging. By creating a bot with controlled access to service logs and metrics, and the ability to cross-reference its findings with code, our platform team created a good first responder. @oncall-bot isn’t the smartest bot in the world, but it does the first level of investigation that you’d do anyways. It often gets close enough to the root cause that the rest of the incident management goes much faster. Other times, it doesn’t uncover the issue but still provides a good jumping off point for investigation.

We’re starting to explore more automations like fixing common on-call issues, but we are early in this part of our journey.

Tooling is easier to build than ever

For years, testing out mobile builds at Hinge was an onerous process if you weren’t a mobile engineer. There were two approaches. If you wanted to install Xcode and Android Studio, you could build the app locally on the right branch and then run in an emulator. But many people who needed to test, like QA analysts, weren’t comfortable managing Git and running builds. So we would also publish a build to e.g. TestFlight so that they could test.

Two engineers on our platform team dreamed of a better way. Imagine a Mac app that just listed all of the active branches in both the Android and iOS repositories. In this dream it can automatically install the Android and iOS emulators for you. While we’re writing a wishlist, we’d also add that we could easily publish new builds to it, and it was so easy to use that nontechnical people could use it too.

There was a problem. Neither of them knew Mac development.

OK, I’m kidding. It wasn’t a huge problem because they could lean on LLMs to do the Mac development. They built it. It’s awesome. You double-click a build and the emulator just runs with the right version. Everything is handled behind the scenes for you.

This isn’t the only instance of new tooling cropping up at Hinge. We have a Cambrian explosion of new tools like faster deploy scripts, new ways to test our code, knowledge bases, CLI tools, etc. We had all of these ideas in our head, but we didn’t have enough time to realize them. But now that you can just have an agent chew on it in the background, there isn’t a barrier anymore.

What other problems did we have a few years ago that don’t exist anymore? When I joined, I was given a huge Notion checklist of instructions. Each line of instructions linked to another large Notion page that explained the relevant setup of some small facet of the system. It took a long time and inevitably you’d still lack a permission or three you needed.

Well, those are also the days of the past. Now we have a CLI tool named dev that can set up any number of environments for you. Do you want to program iOS, Android, and backend? Done. What is your LLM of choice and what MCPs do you want to have installed? Done.

It drifts sometimes. Nothing’s perfect. But it’s easy to just point an agent at it and say “fix it.”

We’ve also seen wins from MCP. The most obvious is our notifications platform MCP, which can also be accessed via its corresponding Slack bot. It’s a one-stop shop for notifications use cases. If you’re an engineer that has never worked with notifications, just describe what you need and it’ll explain everything you need to get started. If you want metrics on a running campaign, it can look them up. If you need to investigate why a notification isn’t behaving like you expected, it can help you.

Accelerating design, coding, and reviews

So you’ve researched, you’ve prototyped, you have design mockups, and now you have something that you want to build.

Now the engineering starts.

And this is where there is a devil sitting on your shoulder. He leans over to whisper in your ear. And you already know what he wants to say. “Why can’t you just generate the tech spec from the product spec? And why can’t you just implement the code directly from the product spec? How do you even benefit from the tech spec?”

I’ve personally settled on my own heuristic for whether “slop” writing is acceptable in a particular situation: “Did your org care about this kind of writing before LLMs?” At Hinge, we took a lot of care to craft and write design docs. We wanted to ship good architecture and reasonably good code. We were willing to discuss the docs and iterate towards good solutions.

But on the other hand, little attention was given to commit messages. In fact, I don’t think I’ve ever seen someone asked to rewrite a commit message before it would be approved. So this is reflected in my own work: I make sure the docs are written in my own words, but I let the LLM generate my commit messages. When writing this I feel a lot of conflict from imaginary engineers. “It should all be this one way” or “it should all be another way.” But ultimately I think you need to be pragmatic. Different text fields add different amounts of value. Sometimes it needs to be good enough, and sometimes it’s worth spending extra time to keep revising to get it perfect.

So ultimately, the design docs are all written in my own words, even though they are heavily accelerated by LLMs. I’ve tried everything from “Go straight from description to feature implementation” in hackathons, to “let’s generate the design doc with the LLM and see how it goes,” to “let’s just write the design doc like we used to.” And I find that I get better results when I write the design. I internalize my own designs far better when I produce them in some way instead of just reading them. I think you will find the same thing: taking time to internalize your designs will make you a stronger engineer as you continue to build on your own work in the future. Sometimes you need to slow down a little to go much faster in the long run.

And here’s the cool part: you can have the LLM check your tech spec against the product spec and Slack conversations for drift. You’ll find tons of small drift, where the LLM finds that the product spec and tech spec say one thing, but a Slack conversation indicates that some people believe something else. And every one of those is a potential deadline slip that just got highlighted.

I’ve also found that LLMs are amazing at decomposing design docs into sensible sequenced tickets. I’ve experimented with just having the LLM create all of the backend tickets, and it has worked out. The tickets have enough context that an engineer can easily understand the scope and how it fits into the project. LLMs are also great at picking up these tickets and producing a reasonable first draft.

OK, we have tickets and we’re implementing! And this is where people have their own preferred approaches. LLM coding has ended up a lot like IDEs used to be: everyone has their own distinct styles. Some people love the Superpowers plugin and checking in all the specs, some people love centralized knowledge bases. I personally have my own vibecoded workspace switcher for working on many tickets in parallel.

We also have collaborative ways to work. For example, we have a Slack bot that can author a pull request on your behalf from the surrounding context. We typically use this for small changes that fall out of Slack discussions.

Token spend is capped, but you can easily request more if you provide rationale. The goal is to stop unexpected runaway usage.

There’s no single silver bullet that has made us more efficient. It’s just a collection of lots of small efficiencies. Here’s an example: creating a new service was tedious when I joined Hinge in 2023. It was difficult to understand what pre-existing patterns were modern or obsolete, so you made a lot of judgment calls on the fly trying to pick out the most modern patterns. Then you’d write the Terraform, which had a combination of obsolete or overridden fields that had no side effects, or alternatively seemingly-harmless fields that were unexpectedly important. Typically you’d need several attempts to stand the service up in our development environment, and then several attempts to stand the service up in our production environment, just to work through all of the random configuration issues.

This is completely different when I’m writing this in the second half of 2026. You can simply tell your LLM, “write a new service named servicename. Pattern it after preexistingservice. Connect to serviceone and servicetwo’s gRPC servers, and set up the skeleton of a gRPC service but don’t add any endpoints yet.” and it’ll one-shot it. And then you can go to the Terraform and invoke the relevant skill to create the Terraform for the service. You can just point an agent at both directories and ask it to double-check the work before you send it for review.

In fact, don’t sleep on skills. They are a high leverage way to accelerate your work. We are encoding more and more Hinge-specific functionality into skills and I continue to see it pay off. Here’s another example: today I wanted to test a new set of public-facing endpoints that I’ve been building. So I told my agent to test them with my development user ID. And the agent found a relevant skill for issuing these commands against the dev server. From there the agent understood that it had access to a CLI. This CLI produces tokens that auth the request. And then the skill explained how to find the request format. And before I knew it, the agent had exercised every endpoint and printed a table of results. Then I asked it to exercise the content filter. And good grief, agents can curse with the best of them.

So you’ve written your code, and now you’re ready for code review. But a human isn’t going to read it yet. We have a specialized review agent that should be satisfied before a human reviewer is tasked with looking at the pull request. We liked it for two reasons: first, it has a comparatively low false positive rate. Second, it’s just a legitimately good first-pass review bot. It finds issues that frontier models don’t. It finds issues that the human reviewer doesn’t. But you will also find issues that slip past every single bot. And this is where it’s important that you exercise your own judgment as a human reviewer. Every day, I find problems that the agents miss. The agents can perform incredible feats of engineering, but ultimately they don’t produce the work by themselves. They do this as a way of accelerating you and your organization.

And then when you’re finally finished with coding, I highly recommend running one final consistency check. Prompt your model, “Here are the paths to my backend and client codebases. Here’s the path to the tech spec, read it with the MCP. Here’s the path to the product spec, read it with the MCP. Check MyNewFeature for integration problems between them or inconsistencies between spec and implementation.” It helps find those last issues that would come up in pre-release QA and internal dogfooding.

You can just try things

Hinge has adopted many techniques that have added up to one thing: software engineering is completely unrecognizable from the way it was a few years ago. And there’s a lot of room for more change.

We didn’t adopt everything at once. One feature at a time, we gave them a shot. We tried many techniques and technologies that didn’t work for us. Many sucked for a while until they finally reached a threshold where they became great. But each time you find a working technique, things get a little better, a little faster, or a little safer.

So I really encourage you to keep trying techniques that you hear about. Many have a high skill ceiling and you can get better at them. Many are just distractions that won’t work for you. But the more experience that you get, the more you can differentiate between skills and distractions and continue to improve engineering in your organization.

Thanks to Sammy Spiegel, Jarryd Boyd, Lisa Detig, and Sam Rozenberg for providing feedback on drafts of this essay contest entry.

I built my own Google Docs for my blog

I recently noticed that my blog has fewer than 30 posts, but I've been maintaining it for almost a decade. I'm sure you can guess my next thought:

"This is because I don't have my own hyper-personalized version of Google Docs to use as an editor."[0]

So I set out to fix that. And I have a major advantage: from 2010-2015, I was a Google employee that worked on Google Docs. It's been a while, but I still remember a few things about what makes the ol' girl tick.

It took an 8-hour Q&A with Codex. It took four ChatGPT resets on my $200/mo plan to get it done with Astra on xhigh. But it's done. This is my first post produced entirely by my own custom Google Docs "clone". It powers my blog's editor. It has its own custom operational transform stack. It's slow as shit. I love it to death.

A screenshot of an earlier draft of this post, being edited within the new blog editor

Here is my own personal Google Docs, in all of its glory and horror, from an earlier draft of this post.

To be clear, you're not reading directly from the Google Docs editor's software. The public-facing site and the editor are decoupled. The public site is just static content. The editor accumulates changes and snapshots while I work, tracking everything in a Postgres database. A job is started each time I publish, republish, unpublish, or delete. That job regenerates the static site. That is what you are reading now.

The Q&A from hell

I planned this with GPT 5.6 Sol on xhigh. Astra came out right when I was finishing up the Q&A. Otherwise I would have used it for the Q&A too.

I started the same way everyone starts a big task: I wrote out a huge prompt and hit send.

But I also had one more request: Codex was not allowed to leave any inconsistency or ambiguity in the spec. It had to keep asking me questions until the spec was fully resolved, starting with the most important unanswered question.

It mulled over my initial prompt and cached about 380 questions in a Markdown file. Many of my answers induced Codex to ask several follow-up questions. I didn't do it in a single block. This is just a side project and I have a family and a job. So I did the interview over 4 or 5 days.

Some of these questions were grueling and required long explanations of major pieces of architecture that I wanted. Some required a lot of thought. Some of them were crucial for correctness. Some of them were implementation details that I happen to know about implementing word processors. I settled on a flow that let me gallop through questions pretty quickly.

  • I used voice input and told it "I'm using voice input which might transcribe incorrectly; work harder to try to guess what I originally said."

  • I asked it to ask between 1 and 3 questions at the same time, but it could only ask multiple questions if they were related.

  • And if it could think of an obvious default, it would explain that option and then ask, "agree?" So if all three were okay I could say "agree to all." Usually I needed a hybrid approach like "one: agree. two: do this instead. three: yes but with a small modification."

By the time I got to the last 50 or so questions, I was basically just saying "Agree. Agree. Agree." to each triplet because all of the decisions were inconsequential.

What I asked for

I wanted the static site to have feature parity with my existing blog. For historical context, my blog was WordPress for a long time. In February, I converted it to a Golang server that I host on DigitalOcean. The existing blog needed only very basic features for full parity: headings, text styles, links, lists, images, inline code, block code, quotes, footnotes, embedded YouTube content. Nothing too crazy.

I also wanted my favorite feature from Google Docs: magic redo.

"What is magic redo?"

I'm glad you asked! Go to doc.new and copy/paste this post into the editor. Select some text and bold it. Now scroll down and select some other text (without doing anything else in the meantime!) and type the Redo keyboard shortcut (of undo/redo fame). The text in the new location is now bold. Select some more text and type redo. That text is also now bold. And that is magic redo! It executes the last action as if it was done with the current selection model. It's useful when you want to do the same thing a lot, like adding a few columns to a table. Magic redo has some implications on the backend design, because it means that everything the editor does needs to act on f(selection).

Where I compromised

The biggest compromise was in the view layer. When I worked on Google Docs, it did a lot of clever things to try to make the editor as efficient as possible. They also measured every change to guard against regressions. After I left, they later switched rendering to canvas, which I understand was a huge performance win.

I did not do any of this. I just have it update the DOM node of the paragraph it's editing. I'm not doing any clever tricks. I don't have performance guardrails yet.

I also probably could have just pointed an LLM at its obfuscated source and said, "reverse engineer every feature this editor has" and wound up with a fairly complete feature list. But then it would have generated an editor that did not suit the needs of my blog.

I don't have any accessibility or right-to-left text support. If I ever go blind or move to Jordan I'll yell at Claudex until it makes it happen.

I don't have offline support. No suggested changes.

When I joined Google, Docs was missing what I now consider an essential OT command: "move text". A lot of operations in Docs were implemented with "delete the text here and re-add it here," which could transform poorly against incoming commands and blow them away (since they'd get applied to the range that got deleted). I am not using it at the moment, but I have plans for an "outline" feature that allows me to reorder lists very quickly, and I plan to implement that with move.

"But Jake, you're the only editor. Why do you need to care about how stuff OTs?" Buddy, I implemented my own Google Docs instead of editing a Markdown file. We're beyond logic here.

I've also been experimenting with a "proof of human" feature, which is effectively a read-only Etherpad that I'd link from my posts. Since I have the full command history, every essay could have a complete playback to show that I wrote it. I don't feel completely comfortable launching it, because my writing is VERY exploratory. I write a lot of things that I don't mean which I later delete. I don't want people to be able to screenshot it out of context.

The implementation

I implemented the project using GPT-6 Astra on xhigh.

I have a $200/mo ChatGPT account for my side project coding, and needed four resets to implement it. But I had one advantage: the week I implemented it was an incredible week for resets. So huge shoutout to Tibo. I started right after my weekly quota refresh and already had three banked resets available for use. And there was one more unbanked reset. The initial implementation used 4 full weekly quotas and almost finished the fifth before it finally finished.

I made some basic mistakes in prompting that came back to bite me. A lot of time was wasted on testing and verification. I don't remember exactly how I prompted the model since I did the Q&A a month ago. But I'm sure that I said that it needed to verify everything, that it needed to test all cases, and stuff like that. I think I also suggested that it try fuzz testing on the editor to test out the OT stack.

I made the mistake of acting like I was describing a design to a senior engineer. But no, Astra is not human and does not have a shared human understanding of what I mean when I speak inexactly. So it handled these in the most robotic way possible. "Fuzz testing" turned into a series of 10,000+ case integration tests that needed over five hours to run each. Each day I'd tell it to delete a massive test and replace it with the minimum number of test cases to achieve equivalent case coverage, and each night it would birth another massive 10,000 case integration test that took 5 hours to run. I'm sure this burned a massive number of tokens. Eventually I banned it from generating tests with more than a few hundred cases. It still mentions this fact in completely unrelated situations. This always makes me wonder, "..... but were you actually thinking you might do that?"

Telling it to "verify" things was also a mistake. The app was unusably slow at first. It added runtime checks that choked the app to death.

For background, when you have a full operational transform stack, many features need to be transformed against incoming changes. If you have an undo and redo stack, everything in the undo and redo stack needs to be modified so that they can still apply cleanly. If a command bolds the letter 'b' and a letter is added before it, the indexes referenced in the command need to shift forward so that 'b' is still bolded if it gets applied.

In this case, I requested that grapheme clusters always had to be valid when edits were applied. To me, this means "after application of the edit but before sending the commands to the server, check the surrounding model context until a safe boundary to ensure that the operation did not result in an invalid character sequence." But the generated app would "verify" the undo and redo stack on every single character press by making a copy of the document, applying the edit, and then running a full-document grapheme cluster check. So typing a few characters into the editor was enough for it to stall and stutter.

I've done a few rounds of performance work at various granularities, like "walk me through everything that happens when I press a key" and "optimize typing 1000 characters", and I think I've eradicated most of the unnecessary work. Typing is relatively smooth, which is nice given that I did not optimize the design for performance.

The cutover

Since the project involved migrating from the static site to the hybrid site (dynamic backend, static frontend), the cutover became an Epoch event of religious significance to Astra. Astra was obsessed with the cutover. My Makefile and Ansible implementations became littered with a ton of extra commands and checks. Not only support for the old version and the new version, but tons of verification checks to ensure complete site sanity at every intermediate step of the cutover. And Astra needed Verification, which it got by hashing lots of different things and calling it Proof. So the entire thing was Verified with Proof, which was very comforting.

Since my site was approximately 30 Markdown files that were backed up on GitHub, the stakes could not be lower. I decided to just try Astra's cutover. If it failed I was just going to recreate the droplet and have it start fresh. But it did a fine job. Migration successful. The site works great.

But then the cutover continued to exist. Every change needed to be tested for compatibility with the pre-cutover and post-cutover and intermediate steps. It still asked me about how I'd like to handle the legacy Markdown site when describing new features.

So obviously I eradicated the cutover. I tasked Sol with the cleanup. It deleted the old Markdown static site as well as the Go code for serving it. It also eliminated all of the LLMslop guardrails: the cutover mentions, verification checks, all Proof and Evidence of Verification. All of it gone. And as a bonus, I haven't seen it mention the cutover in a while. Codex has truly forgotten about the cutover.

How do I edit it now that it's live?

First, I have a bunch of skills that manage each stage of the process. File a ticket (and give me a Q&A over any ambiguities), start a demo server, clean up and submit, etc.

I have a vibecoded workspace switcher, lovingly called ngmi, which manages all of my agents and how they interact with the project lifecycle. When it picks up a ticket, it automatically creates an agent, blocks on unmerged dependencies, has a merge queue to serialize the submission of finished tickets, etc. It runs on a separate DigitalOcean droplet, and can handle 3 or 4 agents running at once without struggling too much. This would be too limiting for my day job, but is fine for a side project. I run it on a DigitalOcean droplet since I work from both my Windows gaming PC and my MacBook M1.

ngmi can spawn its own non-task agents and has its own MCP, so I can also command it to do things from within the agents managed within ngmi.

Since I love taking my side projects too far, I have my own ticket tracking program with its own MCP at bitlogwork.com. Whenever I want to change something, I just file a ticket and ngmi automatically picks it up. It detects what part of the stack I'm working on (ticket tracking, analytics, the blog) and starts a demo server when it completes the task. It can generate and serve explainer HTML pages to highlight different aspects of the change that I'm curious about. I can switch to the agent and ask for changes.

I've been doing performance work and adding features over the past month, as I have energy to do side project work. It's finally in a position where it's not so painful to write a post, so here it is! The inaugural blog post coming from my own personal Google Docs clone.

Lesson: reality has an infinite amount of detail

One-shotting is for losers. Find ways to uncover all of the hidden spec that matters without getting bogged down in tangents that don't matter.

If you tell an LLM "Make a modal dialog with options OK/Cancel that asks if you're sure you want to delete this before executing the delete action," you may think that you very exactly specified what it does. You can probably already picture what it looks like. But the LLM will guess an insane amount of spec behind the scenes. Where is it positioned? Do the corners have border-radius? Does it have a border? Does it have a drop shadow? What is the padding? Does it need a separate title or does the raw question text suffice? What order are the buttons? What color are the buttons? What is the exact copy? It just goes on and on. There is no limit to how detailed the spec can be.

But when you're building something and you care about the output, there are an infinite number of possible details that matter. There are a lot of types of implementation where you can just shut off your brain and allow the LLM to pick something for you. But if you really want to produce a great product, you still need to have vision and taste and take control of the output. Take it from me, the guy who does not have vision or taste but really wanted to take control of the output.

Lesson: software isn't a moat

Any of your old employees can recreate large swaths of your product in a few days of motivated work (or a month or two if they're kinda just winging it on the side). It's all about ecosystem, partnerships, and all of those other intangibles that actually require hitting the pavement, working the phones, hunting through your network, and doing all of the other grueling work of building up your place in the world brick by brick.

Footnotes

[0]

You might say, "But Jake, you do have your own copy of Google Docs. Google Docs is free. You could simply use it and copy-paste the content." But I turn that back around to you, and ask, "Is it really mine? Do I have the freedom to make the real Google Docs this underfeatured and unperformant?"

How to cure AI psychosis

To any afflicted CEO in my readership: I want to cure your AI psychosis.

⚠️ First, a disclaimer ⚠️ This post does not discuss legitimate medical issues; I'm not a doctor and none of this is medical advice. Talk to a real medical doctor if your mental health is in question.

As a quick aside, I'm experimenting with recording my blog posts for YouTube. So if you'd rather see that, here it is:

View on YouTube

Instead, let's talk about the fun kind of AI psychosis. The kind where people see LLMs as a genie-in-the-bottle, able to grant any wish.

Aaron Levie, the CEO of Box, recently tweeted that CEOs are uniquely prone to this kind of AI psychosis. He says that CEOs do not perform the "last mile" of work. They never see the effort required to coax LLMs to perform useful work. They just see the happy path. "Here ya go boss! I made this prototype with an LLM prompt, just like you asked."

If you're a CEO and you're working daily with LLMs and you have a good sense of their limitations: I hope you're having a good day! You can close the tab. But for the rest of you, buckle up, because we need to talk.

Anyone riding the NYC subway these days can observe this themselves. The ads plastering the trains are preposterous. They make promises like, "make an entire presentation with a single prompt." Obviously this is disingenuous. But there must be some subway riders influenced by this, at least enough to visit the website. And those people must somehow be responsible for inking SaaS contracts. And you might wonder, "Why can't it? Why can't it make a whole presentation with a single prompt?" Let's save that question for the end of this post. We can decide together whether it's just "a single prompt" or there is an incredible amount of hidden work.

The answer has to do with two questions.

  1. How easily can the LLM supplement its context?

  1. How quickly can the LLM verify its output?

But we'll talk about that later.

Let's go back to the beginning.

At release, ChatGPT could do things that no computer had ever done before. You could type any request you wanted, and it could do it. Need to write an email to a business? ChatGPT can do it. A poem about your friend in the style of Walker, Texas Ranger? Giddyup partner.

So why didn't the entire world lose their jobs at this point? Because LLMs were absolutely useless for real work. They confidently lied, over and over. They fabricated citations. They became legitimately unhinged if a conversation extended more than a few rounds of back-and-forth. They couldn't follow simple step-by-step instructions without getting lost. It generated code that wouldn't even compile, or used imaginary APIs on imaginary libraries. They couldn't count the letters in a word. They failed at trivial reasoning exercises. Sometimes their rudimentary safety checks prevented you from doing mundane tasks, but other times the LLM urged people to kill themselves. And that's an incomplete list of their shortcomings. They had some utility, but the outputs were largely a novelty.

But LLMs have improved so quickly over the past few years. Each of the above problems have better mitigations, and LLMs have gained new capabilities. They got chain-of-thought reasoning strengthened with reinforcement learning. They can call tools. There are now protocols that allow LLMs to interact with your data in any application. Every few months they get model upgrades that supercharge its abilities.

And they are world beaters. They are solving novel math problems that humans have been trying to solve for decades. They can perform complicated research tasks and produce convincing summaries of the results. They beat software engineers in coding competitions. And by all accounts, they can write college-level essays.

So I don't blame anyone for drawing the trend line from these capabilities and saying, "if an LLM can perform high-level mathematics, or implement a complicated step-by-step pull request, then surely it can do any job."

But, I have to level with you: in all that time, with all of those improvements, they have gone from "useless for real work" to "somewhat useful for real work. They cannot produce a final product unsupervised if the quality matters at all.

They will save some time if you are skilled at using LLMs. But LLM use is a skill, and it's a skill with a low skill floor and a high skill ceiling. An unskilled person will take longer to produce a professional-level output than if they had done it themselves. You can also be a LLM veteran and do something that worked before, and it just produces a worse result this time. Unlucky.

Panel 1: How CEOs think LLMs work - almost all time is saved except the part the LLM runs. Panel 2: How LLMs actually work. You have to assemble context, run the LLM, and verify its output, and you do save a little bit of time but it's not almost the whole task.

This diagram isn't meant to be pixel perfect. But I stand by the idea; for each LLM prompt, there is some intrinsic amount of context building and some intrinsic amount of verification. And if the output is too far from the desired result, then you're likely losing time.

If you want to break outside of the tech sphere, go talk to a competent lawyer about how useful they find LLMs. They're great, they save so much time! You can summarize really complex documents and generate briefs and do all of these other things! But you need to check every single thing that it outputs. You don't want to build your argument on a hallucinated case or an incorrect summary. You don't want to email an opposing council LLM slop.

What the hell? Why can LLMs solve math problems, but you can't rely on them to build a legal case without obsessively double-checking everything they do? Why can an LLM beat software engineers at a coding competition, but it can't build a simple presentation, a task that many people might delegate to an intern?

Let's revisit the two questions I wrote above. They explain the whole story.

  1. How easily can the LLM supplement its context?

  1. How quickly can the LLM verify its output?

And let's look at this through the lens of "Build a presentation with a single prompt." Let's really consider everything that you would need to build a successful presentation.

How easily can the LLM supplement its context?

Ok, what context is required to build a presentation in a business setting? Let's say that you're building a pitch deck for a new product offering, and you want potential customers to buy. Your customers might want to know…

  1. What problem they are currently experiencing.

  1. What value the new product offering delivers.

  1. A high-level overview of how the product works.

  1. Data that reinforces your story.

  1. Industry research explaining how similar companies solve this problem.

  1. Social proof from someone that tried the product.

  1. The story about how they successfully migrate to the project.

  1. Any offers, trials, concierge services, etc that make it easier to perform a pilot project for the service.

To build a successful presentation, you might need to additionally consider…

  1. How long is the scheduled time?

  1. Is there anything that would piss off your investors?

  1. Is there anything that would piss off key stakeholders?

  1. Do you know something about the audience that should be incorporated into the presentation, like their tech stack, stated preferences, etc?

  1. Do you have the ability to request that new art assets be made, or are you stuck working from some pre-canned set?

You also might not want them to know a few things.

  1. That you'd accept a steeper discount if they said they were trying a competitor.

  1. There's a feature that you're pitching that is still technically a roadmap feature and not code complete.

  1. Everything is a giant shitshow and we're just keeping it together for public appearances.

And the list goes on. So can the LLM easily supplement its context? Obviously not! The LLM might need to snake its tendrils into every single data source that your company has, including the brains of people who work there. And at a large organization, you will have so much data that it cannot fit into the context window of an LLM. So now the LLM needs to solve the open research question of "how do I find the exact right context that I need to answer all of these questions?"

How quickly can the LLM verify its output?

How do you need to verify a presentation that you didn't write?

First, you need to flip through the presentation. Look for any weird outputs. Are there overlapping regions? Things that are a little too close? When you put two images next to each other, do they visually clash in some way? Some of this could be handled by the LLM but some of it is just the human experience.

Then you need to consider the structure. Is this a compelling pitch? Is this ordered correctly? Should any slides be omitted? Is anything missing?

Then you need to nitpick the content. Are all of the sentences correct? Is it using good word choices throughout the presentation? Did it misunderstand any of the source material and needs to be reworked for accuracy?

And finally, you'll want to actually give the presentation slide-by-slide. Does it all flow naturally when spoken out loud? Are there any unnatural moments? Does anything need to be restructured or reworked?

So the LLM cannot really verify its output at all, so it can't iterate and recurse on the problem. But there are aspects that the LLM might be able to assist with, like finding source documents, answering questions about data that it can access, and generating the slides. So it's very likely that you could build the presentation faster, but there's also a tremendous amount of hidden work involved: uncovering the full context of your problem, what you need to present, and being a strong advocate for the audience in mapping the LLM output into a convincing presentation.

The cure for LLM psychosis

The cure is simply the ability to find the hidden work.

If the LLM can add to its own context via simple searches, or even perform its own logical thinking like building its own lemmas and theorems when investigating a math problem, then the problem is in the LLM's sweet spot. It doesn't need people at all, it can just churn by itself! However, if it needs to sort through more information than it can access within its own context, then the LLM is going to need extra assistance and there is hidden work.

If the LLM is working on a programming problem and can simply write its own unit tests and run the program on sample inputs to verify its own outputs, then you are in the LLM's sweet spot. However, once verification starts to have complex causes like "how do human beings feel when they view the output?" and "Does this otherwise-correct change introduce unwanted emergent behavior when run in the production system?", then there starts to be a lot of extra hidden work for getting the LLM to exist within an ecosystem.