Compaction Is Like Dreaming
At the end of a long day filled with adventures, detours, ups and downs, we get exhausted and our mental capacity declines. There are only so many things, so many threads, we can keep open in our minds at the same time, and so many thoughts we can carry on in a day before running out of steam and needing to take a nap.
Dreaming is how the brain cleans up its accumulated "stuff" in general: flushing accumulated molecules of the wrong type from the day's chemical processes; taking the day's ideas and organizing them, filing them to memory, processing them, revisiting old and recent worries and hopes, whatever is on our subconscious.
I think dreaming is like compaction with AI agents. LLMs have a finite context window: they can only hold so many thoughts in context at the same time. And as their context window fills up, their thought processes start to degrade. They become sloppy. They make simple mistakes. They omit things they ought to know. As of today, compaction is a process that usually takes whatever is the accumulated context of an LLM, and summarizes it into a smaller context that hopefully leaves enough breadcrumbs to continue what was going on before, while clearing up most of the noise of the accumulated day's thoughts. Not every word said or heard in a day has equal usefulness, and not all of it needs to stay in live memory. Humans also have a finite context window. Sleeping and dreaming are probably also related to "human brains approach saturation every ~16h". Social exhaustion, emotional exhaustion, cognitive exhaustion. Our performance declines, our patience runs out, we need to rest to recover our steady-state capacity.
Humans have various levels of memory. We don't keep everything in the foreground of our mind: only the most pressing issues, information we need soon, or things that freshly happened to us. We then file (subconsciously) things to our long term memory. Some sort of two level memory hierarchy is going on. I'm not an expert, but the overall idea seems to fit this overall pattern. And then for things that don't fit our internal memory's capacity, we use tools. We write books, we write text somewhere, we talk to other people and tell them our stories. And we rely on the external world to keep some of our memories for us. We outsource our daily planning to our phone's calendar app. We store years and years of memory in email. We write stories and epics in drawings inside caves. We encode navigational maps of the Pacific Islands in songs. And we mostly keep a rough index of "if I ever want to remember about the car I had in 2012, I should search in my old pictures". We don't store the facts, we store vague information about how we would go about retrieving the information if we needed it again. Like search, like RAG.
So I think the same way of compacting and storing facts is how AI agents should work. Rely on the world, rely on others, rely on externalized recording systems. And like humans don't die every night, agents probably should be able to maintain 100-years-long sessions without having to start fresh every time. They just need to dream more like humans, maybe.
Note: LLMs, AIs, agents... don't flush molecules, they simply organize their thoughts when they do compaction. Maybe they also need to flush something? In the case of humans, the "flushing" is about taking care of the accumulated byproduct of "being an awake brain". How would that map to the physical way by which AIs execute? An open thread to mine for inspiration.