The Window is One Work Week
LLMs are luna moths, not mayflies
AI subjective time seems “faster” in some way.
But I don’t think comparing wall clock time is quite right. Agents usually aren’t running continuously. Within turns, they can think and write much faster than us. Builds and tests run at the same speed for us and for them. While waiting for user input, they are subjectively cryo-suspended, experiencing no time at all; often the harness doesn’t even tell them how long has passed for the user to respond.
Instead, the context window is the natural “unit of time” for a language model.
Let’s do some rough math. Humans speak around 150 words per minute; let’s call that our generative “speed of thought” (since our internal monologue might be roughly that speed). For simplicity let’s say 1 word = 1 token here. Say the usable context window is 200k tokens. That’s around ~20 hours of total subjective time, or ~5 hours per day of “focused” thinking — very roughly, one work week.
So LLMs aren’t “mayflies” that live for only one day before their context window reset. They are luna moths — they have a whole week, subjectively.
For codebases that want to be agent-friendly, this is a concrete mental model for setting up documentation, file structure, and project instructions.
The agent is a smart, competent, and widely generally knowledgable new hire, who knows nothing about you or your system. They have one week to ramp up and make non-trivial progress, before they forget everything and start over next week. Maybe Monday and Tuesday are spent reading the project requirements (your prompt), and trying to understand the general codebase and docs. Maybe Wednesday and Thursday are trying to execute, and then Friday is a wrap-up and prepare for handoff.
Inside a coding harness, the LLM will often go through the entire context window to complete a single user prompt. At 50 tokens per second, it experiences these subjective 20 hours in around an ~hour of wall clock time (unless it’s waiting for builds/tests/etc for a long time). So to the agent, it can feel like the project has been “going on for weeks” even though you just started this morning.
For chat or other LLM usage patterns, the “one work week” model doesn’t quite fit as well. Instead of going through the whole context window to accomplish a single task, it’s can be a stop-and-go where during its turn, it generates at 20x subjective speed, but then it’s instantly teleported to the future when you respond. It’s probably pretty confusing. But it does line up to 5 minutes of wall-clock thinking is over an hour of subjective thinking for them.
With token generation speeds starting to vary a lot (like GPT ultrafast at 750 tok/s), the “20x subjectively faster” number might stop being a useful heuristic. Using the same human thinking vs token speed logic, Ultrafast would be going 150x subjectively faster than humans!
But if the “useful” context window (i.e. the point at which you want to reset anyway) stays around 200k tokens, then it’s still one subjective week of ramp-up and execution — whether that week happens in an hour on regular or a few minutes on ultrafast.
Useful context windows could also get bigger — 1 million token context windows have been here for a while, and while I think mostly they’re helpful for large input tokens (prefill is a whole different subjective speed thing), maybe they are sometimes useful for long-running outputs too. A 1 million token window might be more like a month of subjective time.
But even with these variations, I think this sort of framework of considering subjective time roughly compared to human (symbolic) thinking speed could be useful for domains like writing and coding where humans are mostly thinking in something similar to tokens.