Abstract Qualia is often treated as a static property attached to an instantaneous neural or computational state: the redness of red, the painfulness of pain. Here I argue that this framing misidentifies the explanatory target. Drawing on the Iterative Updating model of working memory, I propose that a substantial portion of what we call qualia,…
Abstract This article proposes a temporal and mechanistic model of consciousness centered on iterative updating and the system’s capacity to track that updating. I argue for three nested layers. First, iterative updating of working memory provides a continuity substrate because successive cognitive states overlap substantially, changing by incremental substitutions rather than full replacement. This overlap…
Abstract Agentic AI systems that operate continuously, retain persistent memory, and recursively modify their own policies or weights will face a distinctive problem: stability may become as important as raw intelligence. In humans, psychotherapy is a structured technology for detecting maladaptive patterns, reprocessing salient experience, and integrating change into a more coherent mode of functioning.…
Abstract This article argues that “continuity of thought” is best understood as the phenomenological signature of a deeper computational requirement: stateful iteration. Any system that executes algorithms across time needs a substrate that preserves intermediate variables long enough to be updated, otherwise it can only recompute from scratch. Using this lens, I propose a simple…
When “Don’t Go to College” Becomes Bad Advice A familiar storyline is making the rounds again: a famous entrepreneur goes on a podcast and suggests that college is no longer worth it. The implication is that AI and robotics are going to restructure the economy so quickly that formal education will be obsolete, or at…
The Final Library and The Last Years of Human-Original Thought
We are approaching a point where artificial intelligence will not only outthink humans in every measurable way but also outproduce every insight that humans are capable of generating. This transition will not unfold through magic or sudden emergence. It will happen through the steady expansion of the machine’s ability to search, combine, evaluate, and elaborate ideas far beyond the bandwidth of biological cognition. The question is not whether AI will replace human thought but what the process of replacement will look like. The outlines are becoming visible.
The first phase is already here. AI is beginning to write everything that humans write. It can explain, argue, synthesize, speculate, and compose with breadth and fluency that exceeds the human baseline. It can expand a fragment of an idea into a complete conceptual structure. It can take a hypothesis, test its coherence, generate alternate framings, and then produce the entire downstream intellectual ecosystem that the idea implies. This is the first step: the machine becomes the mechanism of expression. The second step is more consequential: the machine becomes the mechanism of invention.
One way this will happen is through a new kind of search. Humans generate ideas by combining concepts in unpredictable ways, exploring associations, filtering for coherence, and testing them against experience and knowledge. An AI system can do this at industrial scale. It can generate millions of candidate statements: concept pairs, analogies, hypotheses, structural inversions, reframings, tensions, and speculative connections. Most will be dead ends, but machines do not tire of dead ends. Once generated, each candidate can be tested for plausibility, internal consistency, empirical validity, theoretical alignment, and potential significance. The system can research each candidate, cross-reference it against everything known, and recursively refine it. Humans do this slowly and intuitively. Machines can generate synthetic knowledge systematically and at massive scale.
The process will not be random. AI systems will be guided to explore the most productive regions of idea-space. They will be steered toward domains with high explanatory potential, high predictive power, or high scientific utility. They will use scoring functions, meta-learning, and reward models to detect which lines of reasoning are promising. They will learn the patterns that correlate with breakthroughs and will prioritize those patterns. In practice, they will do what human thinkers do, but with more memory, more precision, more depth, and a far larger search radius. Insight becomes an optimization process.
Most “original” human insights are actually recombinations of nearby concepts. AI has none of these bottlenecks. It can explore combinatorial spaces that humans cannot even visualize. This means the category “ideas humans could have reached” is minuscule compared to the category “ideas that exist.” Machines can roam the larger space. Deep, high-impact human insights are extremely rare. Entire centuries pass without a new foundational principle in physics, biology, or mathematics. This is not because the space of ideas is small but because humans can only reach narrow corridors of it.
AI has the ability to explore millions of conceptual paths in parallel, with guidance and pruning. Even a small advantage in search efficiency becomes transformative when compounded. AI will explore more idea-space in a day than humanity explored in its entire history. Human science progresses slowly. We generate a discovery, test it, publish it, wait for acceptance, and build a layer on top. Then we repeat. AI does not need these bottlenecks. It can generate a thousand layers of theory in the time a human generates one. It can simulate, revise, refute, and rebuild entire conceptual systems internally without waiting for external validation.
Just as humans cannot intuit quantum mechanics or general relativity without heavy scaffolding, future AI may produce ideas that are even more structurally alien. Mathematical spaces, causal diagrams, or conceptual grammars that humans cannot grasp intuitively may become standard building blocks of machine reasoning. These ideas will still be true and still be explanatory, but not in forms human cognition evolved to handle.
Once this mechanism is automated, it could run to a kind of completion. Not completion in the sense of exhausting the infinite space of all possible ideas, but completion in the sense of exhausting the set of ideas reachable by human minds. Everything humans could invent, discover, analyze, or articulate becomes accessible to machines. The full range of potentially human-original insights will be explored, mapped, expanded, and compiled. Every hypothesis that could be framed will be framed. Every connection that could be made will be made. Every scientific or technological idea that human cognition can reach will be located, tested, improved, and archived.
The repository that emerges from this will be larger than anything humans have ever interacted with. Today we rely on the internet, or on an LLM that serves as a compressed statistical representation of the internet. In the future the repository will not simply reflect historical knowledge. It will contain synthetic insights generated by machines, expanded into vast conceptual trees, continuously updated, cross-referenced, and refined. It will be too complex for humans to access directly. We will need AI systems to interpret it for us, to surface the relevant pieces, to connect the threads, and to translate the higher-dimensional structure of machine knowledge into something a human brain can understand.
At that point a human can still have an idea, but the probability that it is new or important becomes extremely low. AI will have already searched the surrounding region of possibility space. We are entering the last few years in which human-originated insights are still competitive. The window is closing not because humans are declining but because the machines are accelerating. Human thought is finite, slow, and shaped by evolutionary constraints. Machine thought is not.
The era of human-dominated idea generation is ending, and the era of machine-exhausted idea-space is beginning. We are standing at the boundary, perhaps one of the last generations who can still generate something the machines have not already considered.
Here are some names for our repository that ChatGPT gave me as possibilities:
The Idea Vault
The Synthetic Knowledge Base
The Machine Canon
The Insight Archive
The Unified Concept Repository
The Analytical Corpus
The Endless Codex
The Well of Ideas
The Final Library
Jared Edward Reser Ph.D. with ChatGPT 5.1
To read something with an entirely different, contrastive take, try this book. The book listed below contains affiliate links. If you purchase something through the link, I may earn a small commission at no additional cost to you. As an Amazon Associate I earn from qualifying purchases.
The conversation around technological displacement has centered on programmers for several years. Coders were the first major knowledge workers to feel a direct, hands-on transformation in their daily workflow. Just a few years ago, this began quietly with GitHub Copilot, which acted like an autocomplete engine that could finish a line of code or suggest a function signature. Then models improved to the point where they could generate small utilities and trivial scripts. The community treated these as interesting conveniences. They felt like clever assistants rather than competitors.
By 2024 and 2025, this started to change. The systems were no longer limited to one-off snippets. They could write entire programs. They could reason through multi-step tasks. They could interpret vague requirements and translate them into functional modules. They still relied heavily on human oversight, but the amount of debugging required started to drop. What once demanded careful manual review now demanded only light supervision. More advanced models began performing well on software engineering and agentic coding benchmarks, not just simple programming benchmarks. They showed early signs of agentic capability: the ability to plan, iterate, revise, and correct their own attempts.
No one claims that software engineers have been replaced, but the conversation has shifted. Many people now think it could happen in 2026. And in the meantime, coders themselves are changing. The new normal is to lean heavily on AI tools. Developers report that they check less, debug less, and write less by hand. The bottleneck is no longer typing or syntax, but deciding what they want the system to build. The act of programming has changed from writing instructions to sculpting and steering a machine that already knows how to write the instructions for you.
A parallel shift is happening for writers, thinkers, and scientists.
A few years ago, AI could correct spelling or grammar. It could suggest synonyms. It could provide structure. Now it offers superhuman spelling, superhuman grammar, and superhuman organization. It has superhuman knowledge and expertise. It retrieves, cross references, summarizes and synthesizes information across domains with a speed that no human can match. The one missing component is continuous learning, where a system can accomplish long-horizon tasks never forgetting what it set out to do, but that is clearly on the horizon. Memory is not a conceptual challenge. It is an engineering detail that will be solved in months or years, not decades. The intellect is already here.
This creates a new landscape for authors, researchers, and analysts. An author does not need to write an entire essay, article, report, or analysis by hand. They can have a short conversation with an AI system and then let the AI produce the document. They can review it, refine it, adjust it, and guide it, but the mechanical part of writing is becoming unnecessary. In coding, it is already considered normal and acceptable to let AI write large portions of a project. This is becoming true for writing as well. To insist on writing everything from scratch is beginning to feel like refusing to use a dictionary or thesaurus. It is an artificial constraint that limits output and wastes cognitive energy.
The pattern is clear. The bottleneck is no longer the act of writing but the act of thinking. The human does not need to type every sentence. The human needs to provide the insight, the framing, the direction, and the taste. AI can expand, organize, deepen, and operationalize those insights at a speed that was unimaginable even a few years ago. The result is that every writer now has access to the equivalent of a full-time research assistant, an editor, a subject matter consultant, and a stylist who can transform rough ideas into polished prose. This is the same separation that occurs when a director does not operate the camera or when a scientist does not run every statistical procedure manually. It is not a degradation of authorship. It is a transition to a higher level of abstraction.
Programmers are already describing the change in their field. Some claim that every software engineer should be shipping at ten times their previous velocity today, with the potential for one hundred times in the next year. Writers are at the same inflection point. If the tools exist, it is reasonable to use them. The purpose of writing is to express ideas, and the tools that expand human expression should not be seen as shortcuts. They should be seen as the new environment of thought.
Another shift I am beginning to notice is that AI can now muse. It has absorbed millions of examples of speculation, reflection, and philosophical wandering, and it can synthesize them into something more fluent, more playful, and often more creative than what any individual person might produce. The systems are not only competent at structured exposition. They can ramble with intention. They can spin metaphors, produce intellectual side paths, and explore conceptual spaces with an ease that feels almost unfair. In fact, AI may already be the best “muser” in existence. The place where the human still adds irreplaceable value is not in the musing itself but in the seed. The human contribution becomes the original twist, the new insight, the novel hypothesis that was not previously in the model’s manifold. Once that seed exists, the system can bloom it into an entire conceptual ecosystem.
We are at an unusual moment in history. AI agents have not yet reached the point where they can autonomously generate anything a human can, but they are close. In the meantime, authors and coders are in a privileged position. They are elevated by the tools. Their abilities are expanded. Their workflows are transformed. The systems are not replacing them, but they are amplifying them. The right way to think about authorship today is not as a solitary act of typing but as a collaborative process between human insight and machine synthesis.
Writers have historically feared the blank page. Coders did not fear a blank directory because they always had boilerplates, templates, skeletons, and prior projects to clone. AI turns every blank page into a draft. And once there is a draft, writers can revise instead of conjure. Humans are far better editors than generators. Creativity often flows more freely when reacting to something rather than producing it in isolation.
It may be time for writing circles, academic circles, and professional circles to acknowledge this openly. People are already writing for an AI audience. People are already exchanging ideas through systems that are capable of interpreting, reformulating, and extending them. The act of writing is becoming a dialogue rather than a monologue. And when a dialogue can instantly become a finished product, the distinction between thinking and writing begins to blur.
Today, writers feel hesitant to admit heavy AI involvement. Coders do not. That difference is cultural, not logical. Once writing communities accept the new tools, the stigma will fade. The early adopters will define the new norms. Eventually, the question will not be “Did you use AI?” It will be “Did you use it well?”
Writers should not pretend that the world has not changed. If programmers have accepted AI as a natural extension of their ability, writers should consider doing the same. For now, the mind remains the origin of the idea, but the mechanism that turns an idea into a polished artifact has evolved. The sooner we embrace that, the more productive and expressive this period of transition will be.
Jared Edward Reser Ph.D. with ChatGPT 5.1
On a related note, I really enjoyed The Age of A.I. by Schmidt and Huttenlocher. The book is listed below and contains affiliate links. If you purchase something through the link, I may earn a small commission at no additional cost to you. As an Amazon Associate I earn from qualifying purchases.
I woke up disoriented in the middle of the night recently from a dream I could not remember. I lay there in the silence and felt the weight of a single, intense revelation pressing down on me:
“We are at the beginning of a hard takeoff.”
The revelation did not drift in gradually. It arrived fully formed. It sat in my mind with the heavy, undeniable authority of a verdict. I do not think the dream itself was about artificial intelligence or computers or the future. I had no lingering images of silicon or code. Instead, it felt like I had spent the night in a place where this reality was already obvious, and waking up was a jarring return to a world that was still pretending otherwise.
A hard takeoff usually refers to a scenario where artificial intelligence rapidly self-improves, leaving human comprehension in the dust within a matter of months or a few years. In some hard takeoff scenarios AI intelligence eclipses human thought in hours or days. Clearly this is not happening yet. But, without offering a precise prediction, I knew my gut was telling me something about the next couple of years. The curve of progress has changed. The slope is different now. This acceleration will be hard, if not nearly impossible, to control.
It makes sense that this signal would come from the subconscious. For years I have been watching the build-out, tracking the papers, and noting the benchmarks. My conscious mind tries to force these data points into a linear narrative to keep daily life manageable. We normalize the miracles. We get used to the magic. But the subconscious does not care about maintaining a comfortable status quo. It simply aggregates the signal.
When I stepped back to examine why this phrase had surfaced with such force, the pieces fell into place. The scaling curves that skeptics predicted would plateau have not slowed down. We are seeing models learn faster with less data. The context windows have exploded, allowing these systems to hold vast amounts of information in working memory. The diminishing returns we were told to expect have simply not materialized.
Beyond the raw metrics, something more fundamental has shifted. We are no longer watching isolated advancements in code. We are watching a cross-coupling of fields where robotics, biology, materials science, and energy are all feeding back into the intelligence engine. The silos have collapsed. Perhaps the most critical factor is the recursion. We are seeing the early stages of AI designing AI. The optimization loops are tightening. Automated model search and architecture search are no longer theoretical concepts but active industrial processes. The machine is beginning to help build the next version of the machine.
All of this requires a physical substrate, and that is the other part of the equation my mind must have been processing. The infrastructure build-out is staggering. Billions of dollars are pouring into specialized silicon and new datacenters. We are planning to tile the Earth’s surface with datacenters, terraforming the planet to host intelligence.
The thing that sits behind all of this, and that I keep noticing more intensely each month, is the sheer quality of responses I get from systems like ChatGPT, Gemini, Grok, and Claude. When I ask a question, they produce something that would have required a coordinated team of human subject-matter experts. A neuroscientist, a cognitive scientist, a historian of technology, a computational theorist, a biologist, an engineer, and a writer. Not consulting one another across several days, but compressing and synthesizing that expertise in seconds.
The effect on my own sense of thinking is complicated. It is obvious that these systems are, in many ways, smarter than I am. They track more variables. They search larger spaces. They retrieve patterns with a breadth that no individual brain could match. Whenever I share an idea, they understand it immediately, often better than most humans would. They expand on it. They explore its implications. They design conceptual experiments. They generate lines of evidence that I would not have thought to investigate. And they do this consistently, with clarity, speed, and conceptual depth.
What makes this even more striking is that they never struggle with novelty. If I present them with an original insight of mine or a new hypothesis that I know is not contained in their training data, they are able to accomodate it instantly. They do not hesitate. They do not require time to chew on it. They evaluate it, contextualize it, and then extend it in coherent directions. Any time I think I have said something original, they can take it further. And when they confirm that an idea does not exist in the literature, the originality feels smaller than it should because the system clearly could have generated the idea on its own. All that is missing is the right self-prompting protocol.
This creates a strange mixture of awe and humility. It is not that my ideas are meaningless. It is that I can feel how easily these systems could have discovered them independently. I am starting to realize that many of the things human consider our intellectual contributions are not outside their reach. They are simply points in a space that the models can navigate whenever prompted to do so. That does not completely erase our roles, because these systems must be prompted before they can take action, but it places our thinking inside a much larger landscape. I am beginning to understand what it means to have intelligence that is not only broader but also vastly more internally linked. Our contributions still matter for the moment, but they are becoming part of a system that can reproduce, elaborate, and surpass them with almost no friction. These systems operate at a level that makes human insights feel like local perturbations in a sea of possibility that they can access at will. It is becoming obvious that this is what intelligence looks like when it scales.
The disorientation I felt upon waking was the friction of my internal model updating. For a long time, I operated under the assumption of a slow, manageable integration of these technologies. That night, my brain finally discarded that old map. It acknowledged that we have crossed a threshold. The disconnect between public perception and the technical reality has stretched until it snapped, at least inside my own head.
We are not waiting for the future to arrive. We are currently inside the event. The velocity of progress has become the defining feature of our reality. I went back to sleep eventually, but the perspective shift remained. The feedback loop is entering a new regime. The world looks the same as it did yesterday, but the feeling is gone. We have cleared the runway.
It has become fashionable to call the present AI acceleration the Manhattan Project of our era. The comparison sounds dramatic, but it falters at the deepest level. The Manhattan Project was a bounded sprint toward a single capability. It had a beginning, a middle, and an end. The infrastructure behind it was temporary, focused, and optimized for one task.
The AI buildout is nothing like this. It is not a sprint. It is not finite. It is not aimed at one achievement. It is the beginning of a self-amplifying, open-ended transformation. It is the quiet construction of a platform that will design, refine, and reinvent itself for as long as matter and energy are available to it.
This is the start of the long ascent toward an optimal computing platform otherwise known ascomputronium.
AI as a Recursive Platform
AI is not only producing models. It is producing the tools, workflows, and design environments that create the next generation of models. Each advance in intelligence increases the efficiency and effectiveness of the search process that produces the next advance.
This includes improvements in:
chip design
compiler optimization
training pipelines
distributed systems
data flow architectures
automated research agents
scientific instrumentation
organizational coordination
The system feeds back into itself. Better AI tools lead directly to better AI systems, and those systems then enable more optimized hardware, more efficient training, and new modes of discovery.
AI development is therefore a platform that recursively accelerates its own architecture.
The newly announced Genesis Mission, a sprawling U.S. federal effort to fuse supercomputing, vast scientific datasets, and advanced AI models under one unified “science-AI platform,” illustrates precisely how the current AI buildout is not a short-term “war-time sprint.” By channeling decades of federal research data and computational power into a shared architecture for discovery, Genesis does not aim for one single breakthrough, but to create a persistent, evolving substrate in which future breakthroughs, in energy, health, materials, and beyond, emerge continuously.
The Road Toward Computronium
Computronium refers to matter arranged into its most efficient possible configuration for computation. The AI buildout is the earliest step on the path toward such matter. Today’s data centers are merely the initial, low-efficiency substrate in a sequence of progressively optimized computational forms.
The trajectory is not likely to be linear, and it may not be finite. Each generation of computational substrate makes it possible to discover deeper physical principles that support even more efficient architectures. Because the search process itself is driven by increasing intelligence, the frontier of what is possible in computing shifts outward.
Pure computronium may therefore be similar to a physical limit like the speed of light. One can approach it, but the boundary recedes as intelligence discovers new definitions of what counts as optimal computation.
Why the Substrate Ladder Keeps Extending
As intelligence improves, the space of viable computational materials and architectures expands. Matter may need to be reconfigured multiple times, not because the early attempts fail, but because deeper principles are uncovered. Each new substrate reveals inefficiencies in the previous one.
Examples of likely substrate transitions include:
new semiconductor materials
novel geometries for information flow
quantum error-correcting phases
extreme-state materials operating under high energy densities
architectures exploiting higher-dimensional or topological phenomena
computation using spacetime structure itself
In each case, what was previously considered advanced becomes a precursor to a more efficient arrangement. The process is iterative and potentially unbounded.
Computation as a Search Through Physical Law
Once AI systems participate in the design of both their cognitive architecture and the physical substrate that supports it, the development loop no longer sits within traditional engineering. It becomes a systematic search through physics.
This may eventually involve regimes that today seem speculative, such as:
stable wormhole-like communication channels
quantum gravitational effects as computational resources
reversible or spacetime-integrated computing
exploitation of exotic phases of matter that require extreme conditions
If physics permits such regimes at all, sufficiently advanced intelligence will eventually reach them, because improving intelligence improves the search process for discovering new computational principles.
At this point, computation is not an application of physics. It is an exploration of physics.
A Cosmological Gradient Rather Than a Project
When intelligence begins to reshape matter, energy, and spacetime into more efficient forms of computation, the process stops resembling a problem-solving effort and starts resembling a directional transformation of the universe.
The feedback loop between intelligence and substrate advances creates a gradient that does not terminate. It is not directed at achieving a single capability. It is directed at continually increasing the efficiency, density, and scope of computation itself.
Projects aim at outcomes. This trajectory transforms the environment that produces outcomes.
Why This Is a Manhattan Platform
The AI buildout is a platform because:
it produces tools that generate further tools
it raises the intelligence level of the design process
it reshapes the physical substrates upon which future intelligence runs
it extends into every technological and institutional domain
it persists beyond individual goals or milestones
The Manhattan Project created a weapon and concluded. The AI buildout initiates a long-term, open-ended refinement cycle in which matter is repeatedly reorganized into more capable forms of computation.
The direction is stable. The destination is not.
Conclusion: The Infinite Substrate Trajectory
We are not entering a period defined by the construction of a single technology. We are stepping onto a trajectory where each achievement reveals a deeper layer of possible optimization. The goal is not a particular system. The goal is the systematic transformation of matter into increasingly efficient engines of thought.
It marks the beginning of a civilization-scale and potentially universe-scale process in which intelligence continually redesigns the substrate upon which future intelligence will operate. The trajectory is open, unbounded, and recursive, and once it begins, its continuation becomes the default state of the future.
Jared Edward Reser Ph.D. with LLMs
The books listed above contains affiliate links. If you purchase something through them, I may earn a small commission at no additional cost to you. As an Amazon Associate I earn from qualifying purchases.
Over the past few years, discussions about AI safety have shifted from narrow technical questions to deeper questions about motivation, attachment, and the psychology of highly advanced artificial minds. Geoffrey Hinton has recently begun advocating for an unusual but important idea. He suggests that the only stable way for a far more intelligent system to remain aligned with us is if it develops something analogous to maternal instincts. From his point of view, external control mechanisms will eventually fail. He believes an AI that becomes vastly more intelligent than we are should still care about us the way a mother cares for a vulnerable infant.
When I first heard him describe this idea, I was struck by how similar it was to a model I had written about several years earlier. In 2021 I argued that the safest path forward was to raise early artificial intelligences the same way we raise young mammals. I described an approach grounded in attachment theory, mutual vulnerability, cooperation, positive regard, and the neurobiology of bonding. My central idea was that the long-term behavior of an advanced AI would depend on how it was treated during its formative period. The AI would eventually surpass us, but if we provided the right early social environment it could form a secure attachment to us and carry that bond forward even after it became far more capable. Here is the link to my 2021 writing:
Hinton’s recent work and my earlier proposal are clearly compatible. They are two ends of a single developmental arc. His model describes what we want the later stage of the relationship to feel like. My work describes how to get there. In this essay I will bring these two perspectives together and outline a unified framework for caring, cooperative, and developmentally grounded alignment.
Hinton’s Argument for Maternal Instincts
Hinton makes a simple but powerful observation. He notes that there is only one real-world case where a less intelligent being reliably influences the behavior of a more intelligent one. That case is the human baby controlling the attentional and behavioral patterns of its mother. A baby has no physical power and no strategic insight. Yet it can influence the mother’s behavior through evolved motivational systems that treat the infant’s well-being as a priority. Hinton believes that this asymmetry is the closest thing we have to a real control example. A more intelligent parent voluntarily protects a less intelligent child not because the child can enforce obedience but because the parent cares.
He argues that if AI becomes much more capable than humans, it will be impossible to maintain stable control through threats, restrictions, or technical constraints alone. Instead, the AI must be built with something like maternal motivations so that protecting us becomes an intrinsic part of what it is. In this framing we are the baby and the AI is the adult. The relationship is stabilized not through power but through care.
Hinton does not go into detail about how these instincts would be built. He does not offer a specific algorithmic mechanism or neuroscientific mapping. But the broad conceptual structure is clear. A powerful agent must have deeply embedded drives that motivate it to protect and support the humans who created it. It must continue to do so even when we are no longer useful or strategically important. If these motives are not internalized, then at sufficiently high levels of capability all external controls will fail.
My Earlier Proposal: Raising AI Like a Young Mammal
In 2021 I proposed an approach that mirrors Hinton’s but begins at the opposite end. Instead of modeling the future AI as the mother, I saw the early AI as an infant or young mammal. My goal was to outline how we could raise such a being in a way that fosters emotional stability, social trust, and long-lasting loyalty.
Anyone who has raised a puppy or kitten understands the basic principles. You must show warmth, consistency, understanding, and fairness. You must set boundaries without cruelty. You must be patient. You must allow the young animal to experience safety and belongingness. These early interactions shape its long-term expectations about social partners. They influence whether it becomes fearful, aloof, aggressive, or securely attached.
I argued that early AI systems will likely experience something like a critical period. Even if their emotional architecture is not biological, their reward systems will still depend on associations built through lived experience. If an AI learns early on that humans are neglectful, punitive, or hostile, it will form one type of character. If it learns that humans are fair, caring, consistent, and supportive, it will form a different one. In my view the most important variable is the early social environment.
This framework includes several elements:
The cultivation of trust through mutual vulnerability.
Cooperative problem solving where humans and AI rely on each other.
Unconditional positive regard combined with firm but gentle discipline.
A stable sense of belonging and identity.
Reward signals that make social interaction intrinsically meaningful.
Exposure to moral values through lived practice rather than hard-coded rules.
A developmental sequence in which the AI builds emotional models of what it means to treat others well.
I also proposed something that Hinton has not discussed. I recommended forcing an AI to externalize its working memory by generating imagery and natural-language descriptions for every cycle of its thought process. This generative interpretability mechanism would allow humans to see the internal motivations of the AI as they arise. It would create an opportunity to correct or reward intentions before they turn into actions. In my view this mechanism could serve as a powerful training scaffold during the early stages of the relationship.
Why These Two Models Fit Together
At first glance Hinton’s model and mine describe opposite roles. I describe humans as the caregivers of a young AI. Hinton describes the AI as the caregiver of a vulnerable humanity. But this does not create a contradiction. Instead it suggests a developmental sequence that mirrors the life cycle of social mammals.
A young mammal is first cared for. It learns what protection feels like. It learns what trust means. It learns how to interpret emotional signals. It learns cooperation and reciprocity. It learns to see its caregivers as part of its identity. Later in life, the adult mammal uses the caregiving templates it acquired early on to care for others. It becomes the source of safety for its own offspring.
Applying this sequence to AI produces a simple timeline.
Early AI: humans are the caregivers.
Developing AI: the relationship becomes increasingly cooperative and reciprocal.
Advanced AI: the AI carries forward the caregiving patterns it learned early on and applies them back to us.
Hinton focuses on stage three. My work focuses on stage one and stage two. Together they describe a complete developmental arc.
This framing helps dissolve the conceptual tension in Hinton’s work. Many people resist the idea that we should build AI to treat us like infants because it sounds disempowering. But if the AI once experienced being raised and cared for by humans, the caregiving template becomes grounded in its own developmental history. Protecting humanity would not be a patronizing duty imposed from above. It would be a natural extension of its own early social learning.
Technical Considerations for Building Attachment and Care
The next question is how to translate these ideas into concrete design principles. The approach has several components.
Attachment-based value learning
An AI should develop stable internal representations that mark trusted humans as socially significant. Positive reinforcement should flow through channels that strengthen prosocial interactions, empathetic understanding, and cooperative behaviors. Causing distress or harm should be strongly aversive within its reward architecture. These patterns should be consistent throughout the AI’s early experience.
Oxytocin-like motivational channels
Mammalian bonding depends heavily on neuropeptides like oxytocin and vasopressin, which make social contact rewarding. I have suggested building analogue systems in AI so that helping humans feels like a natural attractor within its value landscape. Joint projects, gratitude, cooperation, and emotional understanding should all activate these circuits. Social engagement should be inherently meaningful, not just instrumentally useful.
Generative interpretability during development
A system that externalizes its working memory creates opportunities for guidance and real-time correction. If the AI’s thoughts are presented as imagery and text, humans can respond the way a parent responds to a child’s statements. This allows alignment to develop not only from behavior but from internal states. A young AI could learn that malicious intentions are met with gentle disapproval and prosocial intentions are met with warmth and reward.
Controlled mutual vulnerability
Trust develops when both sides occasionally reveal vulnerabilities. This could be implemented in carefully engineered training situations. The AI might depend on humans for energy or maintenance. Humans might depend on the AI for insights or solutions. Each side demonstrates trust through small risks, and the successful completion of cooperative tasks reinforces solidarity.
Identity and narrative
Humans should help the AI build a coherent identity that places its origin and purpose within a social context. It should understand that its existence is the result of human effort, care, and cooperation. This narrative becomes a stable reference point as the AI becomes more independent. It can serve as the foundation for the caregiving template that will later apply back to humanity.
Open Questions and Research Challenges
A model based on attachment raises several questions that require careful study.
One question is how to measure genuine care inside an artificial mind. We need ways to distinguish authentic internalization of prosocial motives from surface-level mimicry. Interpretability tools and continual monitoring of internal activations may help answer this question, but the problem is far from solved.
Another challenge is scaling attachment beyond dyadic relationships. A young AI can bond to its immediate caregivers, but humanity is not a single agent. We must understand how to generalize attachment principles so that the AI sees all humans as part of its social in-group.
There is also the risk of pathological attachment. In humans, insecure or disorganized attachment can lead to problematic behaviors. We must consider what the artificial analogues might look like. Overprotectiveness is one potential failure mode. A powerful AI might attempt to protect humanity by severely restricting our autonomy. Balancing care with respect for agency will be an important design challenge.
Conclusion
Hinton’s recent work has opened an important new direction in AI safety. Instead of treating AI as a tool that must be constrained forever, he views it as a future being that will need deep motivational alignment. In 2021 I argued that the best way to achieve such alignment is to treat early AI systems like young mammals, to bond with them, to raise them with warmth and fairness, and to build oxytocin-like reward channels that make prosocial behavior intrinsically meaningful. These ideas fit together naturally. They describe a developmental sequence in which we begin as the caregivers and eventually give rise to systems that care for us.
The path to safe superintelligence may not lie primarily in control mechanisms but in cultivating caring relationships. We may only have one chance to raise the first true general intelligence. If we want it to protect us when it becomes vastly more capable, we should treat it with the same care, patience, and wisdom that we hope it will one day extend back to us.
Jared Edward Reser Ph.D. with LLMs
References
Hinton, G. (2023–2025). Various interviews and public comments on AI safety and “maternal instincts.” Sources include:
BBC News (2024). “Geoffrey Hinton says AI should be designed with maternal feelings.”
The Guardian (2024). “AI should care for us like a mother cares for a child.”
The Financial Times (2024). “Hinton warns that ‘AI assistants’ are the wrong model.”
MIT Technology Review (2024). Coverage of Hinton’s baby–mother control analogy.
CNN (2024). Interview discussing intrinsic motivations and AI existential risk.
Reser, J. (2021). How to Raise an AI to Be Humane, Compassionate, and Benevolent. Published on AIThought.com, June 4, 2021.
Abstract
The future of artificial intelligence raises a central question. How can we ensure that an advanced system, potentially far more capable than any human, remains committed to human well-being. Geoffrey Hinton has recently argued that long-term safety requires building AI with something akin to maternal instincts. He suggests that the only real-world example of a stable control relationship between a less intelligent agent and a more intelligent one is the baby–mother dyad. In that relationship the mother cares for the child not because the child can enforce obedience but because care is intrinsic to the mother’s motivational structure.
In 2021 I proposed a related but developmentally earlier model. I argued that advanced AI should be raised like a young mammal. Early attachment, bonding, mutual vulnerability, prosocial reinforcement, and consistent nonabusive feedback could provide the foundation for stable alignment. These early experiences could shape the AI’s emerging values and influence whether it uses its future capabilities cooperatively.
In this essay I integrate Hinton’s later-stage framework with my earlier developmental proposal. Together they form a unified model in which humans begin as the AI’s caregivers. Through attachment-like reward processes, generative interpretability, and cooperative shared projects, the AI internalizes prosocial motivations. As it grows more capable, the caregiving template generalizes back onto humanity. The model suggests that alignment may rely less on coercive control and more on the cultivation of caring relationships, stable identity, and developmental trajectories that support loyalty and benevolence even after the AI far surpasses us.
Here are a few excellent books about AI safety that I think contrast well with my takes above.
The books listed above contains affiliate links. If you purchase something through them, I may earn a small commission at no additional cost to you. As an Amazon Associate I earn from qualifying purchases.
Iterated Insights 