A chapter at a time

I was distilling a book into the vault and one book came out as 347 notes. That is not a slip-box, that is noise. My first instinct was the obvious one: add a cap. Keep the best eighty, drop the rest.

The instinct was wrong, and the reason it was wrong is the part worth keeping.

I was feeding the book to the model in six-thousand-character windows, one at a time. Each window cannot see the others. So each one independently grabs the chapter’s loudest idea and writes it down. When I finally read the actual output instead of counting it, there it was: the same principle, you learn who you are by doing, not by planning, appearing seven times in seven phrasings. The 347 was not density. It was echo. A cap would have kept seven copies of the obvious point and thrown away the quiet, surprising claim at the end of the chapter, the one actually worth resurfacing later in some unrelated conversation.

So the fix was not a smarter number. It was giving the model the whole chapter at once. When it can see the chapter, it states the headline once and goes looking for the parts a careful reader would underline. Redundancy went to zero, and it started catching the sharp, easy-to-miss claims it had been walking past. It read more like a person.

Then I almost made the opposite mistake. If a chapter beats a window, surely the whole book beats a chapter? The context windows are enormous now, a million tokens, and a book is a fraction of that. Just upload everything and let it see the through-line.

I tested it instead of assuming, which is the only reason I caught it. Over a whole book the model front-loaded badly: the back third starved, a few insights for hundreds of pages, while the opening got lavish attention. Worse, it started inventing the quotes. Twenty-two percent of the source anchors it handed back did not exist anywhere in the text. That one is fatal here, because the entire promise of putting a book in the vault is that every insight points back to a real line you can open and check. A confident citation to a sentence that was never written is the single thing I cannot ship.

So the unit landed on the chapter, and it landed there for a human reason. A person does not read six thousand characters and stop, and does not hold a whole book in working memory either. They read a chapter, grasp it, write down a few distinct things, and move on. The right amount of attention for the machine turned out to be roughly the amount a person actually holds at once.

The book’s overarching argument is not lost in this. It is simply a different kind of artifact. The through-line is not one of the cards; it is the note that links them, a hub the chapter insights hang off. The thread is structure, not content, and it deserves its own place.

I keep relearning the same lesson in different costumes. The constraint a person works under is usually not a limitation to engineer around. It is a clue about how the thing is supposed to work. The slip-box came from a man reading one card at a time by hand. It turns out the machine wants to read about the same way.