Idea · Working note

Fifteen agents vs one planner: when specialization starts to cost more than it saves

Notes from building Neocortex — a local-first life OS with too many names on purpose.

Jul 18, 20265 min readWorking noteAgentsHCIArchitecture

Srujan Chidarla

Splitting a personal OS into specialized agents (fitness, DSA, job search, deep work) feels clean on a whiteboard. Coordination tax shows up the first week you actually live in it: shared memory, conflicting goals, and the human who still has to be the router.

The question

Neocortex has fifteen specialized agents and a six-provider LLM chain. The bet was: one agent per domain, one owner per failure mode. The open question is when that bet stops paying rent.

Recruiters do not need a multi-agent paper from a new grad. I still need a clear answer for myself, because the next six months of this project either become a product or a graveyard of prompts.

Where specialization helped

Scope. Jarvis vs Hercules vs Turing do not share a system prompt soup. When fitness advice leaked into job-search drafts, that was a routing bug I could see.

Eval. I can tell whether the DSA agent is useful without waiting for the whole OS to feel “smart.” Small agents are easier to be honest about.

Where it hurt

Shared context. A Tuesday that includes a workout, a LeetCode set, and a recruiter screen is one human day. Fifteen agents do not automatically know that. I ended up rebuilding a thin planner anyway.

Naming is not architecture. Fifteen names look like research. They can also be fifteen ways to avoid a boring CRUD layer and a calendar.

Open questions

What is the smallest agent set that still matches how a day actually feels?
Should the human be an explicit agent in the graph, or only the UI?
When does local-first memory beat another LLM hop for “what did I already decide?”