Why AI forgets your story: the context window, explained for novelists

Somewhere in a novelist's first month of writing with AI the same eerie moment arrives. The assistant that helped you devise your magic system contradicts it with perfect confidence. It renames a minor character. It forgets that your narrator has a sister. The machine is not being careless. It does not know.

Here is the mechanism, in plain language, because this one thing accounts for most of what separates the AI writing arrangements that work from those that fail without ever appearing to.

A desk, not a mind

A language model has no memory in the sense you have one. Each time it responds it reads everything currently before it, which is to say your conversation so far together with whatever documents have been supplied, and composes the next words. That "everything before it" is called the context window, and the metaphor that fits it is a desk. The model is brilliant about whatever pages lie on the desk at this moment and knows nothing else about you or about your book. Not "remembers vaguely." Nothing.

The desk is of fixed size. It has grown enormously, and a modern model holds a great deal of text at once, frequently more than an entire novel's worth. Three things bring novelists to the edge of it regardless.

The first is that conversations are larger than they appear. Every exchange remains on the desk: every draft, every rejected version, every no, try that again. A month of daily sessions on one book comes to several novels' worth of text, the bulk of it scaffolding rather than story.

The second is what occurs when the desk fills. Chat tools cope by discarding or compressing the earliest material, which is precisely where your foundational decisions live. The conversation about the premise goes first. This is why the forgetting appears to single out the things that matter most. Chronologically, it does.

The third is subtler. Even within the limit, models grow less reliable at recalling details from the middle of an enormous pile of text. Stuffing an entire manuscript into every request is expensive, and past a certain point it renders the model worse at your book rather than better.

Then there is the quiet killer — every new conversation begins on an empty desk. The model did not merely lose the early pages. The moment you open a fresh chat it has never met you. Which is why writing a novel in ChatGPT degenerates into a briefing ritual, synopses and character sheets and style notes pasted afresh into every session, forever. We examined that whole failure at length in Can ChatGPT write a novel?

Every AI writing tool is an answer to this

Once you can see the desk you can evaluate any tool by asking what it places there and how it chooses.

A chat window places your conversation on the desk and nothing besides. You are the retrieval system, one paste at a time.

Tools built around a story bible keep a summary of your world on the desk permanently. This is better, though a summary remains a summary. The model knows about your chapters without having read them, in the manner of someone who has read the plot section on Wikipedia and believes he knows the book.

Retrieval on demand, which is the approach Proselon takes, keeps the desk largely clear and teaches the co-writer to fetch. It opens the chapters and story-bible pages a given moment requires, reads them entire, and writes with those before it. Drafting chapter thirty, it reads your outline, the preceding scene, and the chapter in which the promise it is about to redeem was made, as you would yourself. The desk holds what is relevant, fresh and complete, rather than everything, stale and compressed.

That last approach is why a Proselon co-writer at chapter one hundred of a serial still knows what chapter one promised. The desk is not infinite. The co-writer never has to work from memory to begin with, and a continuity ledger keeps the small facts from drifting between fetches — the names, the dates, the eye colours, who knows what.

What to do with this

When the AI contradicts your canon it is not malfunctioning. It is telling you what lies on its desk.

Pasting harder will not remedy that. A workspace that gives the model real access to your book will, and that is the standard to hold any tool to, ours included.

Proselon is free for 30 days if you would like to feel the difference directly, which is the same AI you already use with your whole book on its desk.