Talking to itself

I use two AI coding assistants now. One of them had been feeding my system’s memory for months: every real work session mined overnight for the durable patterns in how I decide things. The other was a blind spot, so I wired it in. Same pipeline, same nightly batch, one more folder of transcripts to read.

Before letting the judges loose on the new folder, I counted what was in it. There were 14,147 session records. Nineteen of them were conversations with me.

The other 14,128 were my own system’s API calls. Every time it distills a book, judges a note, or classifies a reply, it opens a session with the same provider, and the provider dutifully writes a transcript. Each transcript sits in the same folder, in the same format, stamped with the same repository name as my real work. From the outside they are indistinguishable from my life. From the inside they are the system’s own voice, thousands of times over.

Here is what would have happened if I had not counted. The nightly miner mostly dodges them by accident, because a machine session has few turns that look like an operator talking. But the judges had no such luck. One of them decides whether a surfaced memory was useful by reading what the assistant said next in the transcript. The machine transcripts are full of assistants quoting my notes verbatim, because quoting notes is their whole job. The judge would have found its quote every time, marked the memory useful, and fed that verdict into the scoring that decides what surfaces tomorrow. The system would have been grading its own homework and calling it my approval, at seven hundred fake sessions for every real one.

The fix was one boolean. A real session carries the name of the client that opened it; a machine session does not. Nineteen out of nineteen real ones had it, fourteen thousand out of fourteen thousand fake ones did not. I have never met a cleaner dividing line. The walk now checks it before anything downstream is allowed to look.

I keep two things from this. The first is that when a system starts observing everything, the first thing it sees is itself, and its own reflection is the most fluent, most on-topic, most plausible data in the whole pile. Relevance filters do not catch it, because it is genuinely relevant. Only provenance catches it.

The second is that I did not catch it. I had shipped the feature and the tests were green. A reviewer pass whose only question was is this built at the right depth went and counted the folder, and the count was the whole finding. The mirror does not announce itself. Someone has to ask who is actually in the picture.