The screenshot above is a message I see a few times a week. “This session is being continued from a previous conversation that ran out of context.” Everything the model and I worked out over the last two hours got squeezed into a summary, and the summary is now all it knows. Whatever didn’t make the cut is gone.
I’ve been using that line to explain to faculty how these things work, and the more I use it the more I think it points at the real difference between a model and a person. The window size turns out to be the wrong place to look. Your working memory holds about four things at once, which is worse than any model’s. The difference is what happens when you pull something out of storage.
When you remember something, you change it. The memory comes up into whatever you’re already thinking about, mixes with it, and gets written back in the mixed form. This is why eyewitnesses get less reliable each time they tell the story, and why Elizabeth Loftus could convince grown adults they’d been lost in a mall as children. Recall is a write.
A model does the opposite. Nothing I put in the context window touches what it knows. It reads, answers, and the window gets thrown away. The weights only change in training, which for the model is a few months a year, with a notepad in between that gets wiped every night. So we both fail, in different directions. I drift. It stays exactly as it was and then falls off a cliff.
That write-back is where creativity lives, I think. New ideas come from metaphor, from putting two things together that don’t belong together and seeing what survives. Soldering and grading, say, or mycelium and municipal bonds. The mixing that corrupts your memories is the same mixing that produces the metaphor.
You could say the model mixes too, and faster. Ask it for ten metaphors between any two nouns and two of them will be good. So blending on its own can’t be what makes a person creative.
What the model is missing sits on either side of the blend. Before it, the raw material in your head came from a body and a room and a bad week, and the metaphor works because you’ve done both halves with your hands. After it, the blend rewrites the memories it was made from, and then it has to get used, which is where most metaphors die. Papert said we make in order to understand. Making is the selection pressure.
So creativity is blend plus write-back plus stakes, and the model has the first one. It’s also why my students build for a real business instead of a prompt box. The prompt box will hand them the metaphor for free. Living with it is the part they have to do themselves.
