We know a little more about the value of LLMs

Takes about LLM budgeting

Photo by micheile henderson on Unsplash

Tesla caps per-employee AI spend at $200.

Tesla—and other companies—have started limiting AI use at the employee level. The theory behind capping individual users—which I buy—must be that each additional dollar an employee spends on LLMs generates less value for the biz, so limits should help prevent employees from wasting the budget on low-value actions that cost more than the value they create.

They’ve “clearly” read my blog post showing that, left to their own devices, workers use LLMs even for tasks they could do better without them. For example, reformatting documents or generating fancy slide decks are two mostly unnecessary tasks that AI can handle in the background, but it’s probably not worth the high token cost.

---

A sort of obvious LLM budget allocation strategy, if you believe labor and LLMs are complementary (which seems natural to me), is to allocate budget based on the employee’s salary. I’m sure this would go over… horribly, but it makes sense economically.

Like, let’s say total output from a certain worker with productivity parameter A who can spend up to M dollars on LLMs is:

\(M^{\theta} \times A\)

Which, when theta is less than 1, captures the idea of decreasing returns to additional LLM spending as workers spend it on less and less valuable tasks.

Then, we can maximize net output from that worker by solving:

\( \max_{M} M^{\theta} \times A - M\)

Which gives us:

\(\theta M^{\theta - 1} \times A = 1 \implies M = \left(A\theta \right)^{1/\left(1-\theta\right)}\)

So, we should assign a larger budget to more productive workers, and in a competitive labor market, productivity is well proxied by wages. So, we should have a regressive AI budget, with larger allocations to higher-paid workers and lower allocations to lower-paid workers.

I don’t expect many companies to do this directly even though it makes a lot of sense from an efficiency perspective. They’ll do it indirectly, of course, via allocating different amounts to different job families or levels.

---

It’d be cool if a company experimented with LLM budgets. Randomly give some workers more of an LLM budget and others less. Then, you’d get a decent read on the average value of tokens. It’ll be difficult to get a lot of power unless you’re a large company… like Tesla. It’d be cool to get the return on AI spend (ROAIS?). If a company does this, please publish the results. It’d be super interesting how the return on tokens (ROT… okay, this one’s unlikely to catch on) compares to their price in corporate land. The context is pretty important here: mature companies aren’t doing a ton of completely new, green-field projects. Most work is fixing or improving stuff. So, the ROT might not generalize to vibe-coding, etc, but it’d be super useful for companies figuring out what their budgets should look like.

If you’re reading this and need a way to justify doing an experiment like this to your employer, just remember that you can share the results of this experiment with OpenAI or Anthropic during the next contract negotiation and, if the ROT is low, probably convince them to give you massive discounts.

---

For my own personal AI spend trend… I have used LLMs less and less throughout 2026—which is sort of the opposite of how I’ve adopted most new technologies. Usually, as I move further up the learning curve, the tool becomes increasingly valuable to me, so I use it more. But it actually takes a remarkably long time to get, say, a query right with LLMs. I’ve accepted that it’s almost always easier and quicker for me to just write it, even after factoring in having to DM someone about exactly what column ABC means.

I still haven’t really found a way to use LLMs effectively for business-process-type tasks either. They generate unusable, overly dense documents and presentations with far too many images or too much formatting. Give it to folks plain, simple, and true. Write directly. Just say it. Don’t present it. You are hereby banned from using adjectives ever again. Do not editorialize. No matter what prompt you give it, it just doesn’t understand how to Be Normal (tm).

On the more positive side for the tech, I’ll never debug without AI again. It’s so incredibly strong at getting the details right when checking my code that I run basically everything I write now through the prompt “Check this for bugs. Don’t fix the bugs. Just report them.” Life-changing.

I’m not sure how well my use cases generalize, but something I’ve noticed with “generative” AI is how bad it is at generating stuff compared to evaluating stuff. If I give it a claim or some code, it’s very good at figuring out the problems from the context I’ve generated. But if I want it to make a claim or code itself, it’s a mess without significant elbow grease that I’d rather spend just writing the first draft myself and using the robots as the fanciest of spell-checkers.

I currently allocate $0 to LLMs in my personal life, aside from whatever I get from Gemini for having the “premium” Google account, which I mostly have for the extra storage, because Codex’s free tier is far too generous. I debugged this entire side project (about 10-20k LOC Rails app) easily using the Codex free tier.

I feel bad about extracting all that value from OpenAI without paying them a dime. Not bad enough to pay them, of course, but I’ll give them an in-kind payment: free pricing and packaging consulting. You should either make the free tier less generous or just kill it and offer a $5-10 plan with low limits. You’re welcome.

---

Thanks for reading!

Zach

Connect at: https://linkedin.com/in/zlflynn