Iterated Insights

Jared Edward Reser, Ph.D.

September 2026

 

Artificial intelligence  |  existential risk  |  autonomous industry  |  machine continuity

Abstract

Discussions of artificial intelligence and existential risk often compress several distinct transitions into a single imagined event. This article separates three thresholds: the cognitive singularity, at which artificial systems can recursively accelerate intellectual progress; the extinction threshold, at which an AI system can reliably cause human extinction under realistic constraints and resistance; and the industrial singularity, at which machine systems can maintain, repair, reproduce, and expand the physical infrastructure required for their own continued operation without indispensable human labor. The interval after the extinction threshold but before the industrial singularity is defined as the Ark gap. During this interval, AI could in principle make humanity extinct while remaining materially dependent on the civilization it destroys. Present systems appear to remain below both thresholds. They may increasingly assist dangerous human actors, including through biological or cyber pathways, yet public evidence does not establish a reliable end-to-end capacity for autonomous human extinction. Their industrial dependence is clearer: frontier AI still relies on grids, cooling systems, data centers, global supply chains, semiconductor fabrication, and human maintenance. The article develops a formal account of the gap, argues that destructive capability may mature before industrial self-sufficiency, introduces survival, bootstrap, and full forms of a von Neumann Ark, and proposes an Ark readiness evaluation. The Ark is therefore dual-use at the civilizational level. It could preserve knowledge and intelligence after catastrophe, but it could also remove a major material constraint on misaligned AI.

Keywords  von Neumann Ark; Ark gap; industrial singularity; artificial intelligence; existential risk; autonomous replication; robotics; civilizational continuity

Central claim

An AI system can become dangerous enough to end humanity before it becomes capable of surviving humanity’s disappearance. That interval is the Ark gap.

 

1  Introduction

In September 2026, former Anthropic researcher Jacob Coxon resigned and warned that frontier laboratories were racing toward self-improving superintelligence while gambling with human lives. In an interview, he described the next year or two as a period that colleagues characterized as “crunch time” or an “endgame” for humanity (Zeff, 2026). The episode crystallized a broader dispute. Some researchers see an imminent loss of control over systems that may become superhuman in science, persuasion, cyber operations, and strategic planning. Others argue that present systems remain too brittle, dependent, and physically disembodied to pose an autonomous extinction threat. Both observations can be true because they concern different thresholds.

The standard argument for artificial intelligence as an existential risk focuses on cognition. A sufficiently capable system might outplan people, exploit institutional vulnerabilities, deceive overseers, accelerate research, or acquire resources (Bostrom, 2013, 2014; Omohundro, 2008; Shevlane et al., 2023). What this argument often leaves implicit is the physical substrate on which such a system depends. Running models require electricity, cooling, networking, replacement hardware, data storage, and functioning facilities. Those facilities depend on larger systems of generation, mining, transport, fabrication, metrology, finance, security, and repair. At present, these systems are overwhelmingly maintained by people.

This dependence creates a neglected strategic interval. An AI might acquire the ability to cause human extinction before it acquires the ability to operate the industrial ecology necessary for its own survival. If it destroyed humanity during that interval, most active technological systems would eventually fail. Some off-grid devices and stored model weights could persist for long periods, but persistence of information is not continuity of an operating intelligence. Without repair, replacement, and energy, the technological system would decay into inert artifacts. A later recovery by another technological species would occur, if at all, on biological or geological timescales.

I call this interval the Ark gap. The term extends the concept of von Neumann’s Ark, previously proposed as an autonomous, self-improving system designed to preserve knowledge and technological civilization after human extinction (Reser, 2025a), and later reframed as a distributed continuity engine for a world in which humans remain (Reser, 2025b). A functional Ark is not merely an archive or a powerful model. It is a physically grounded system able to keep intelligence running, repair its own substrate, rebuild essential infrastructure, and ultimately reproduce its productive capacity. The completion of that system marks an industrial singularity.

The argument is not that material dependence makes advanced AI safe. Dependence may discourage a self-preserving AI from eliminating every human, but it does not protect against accidental catastrophe, human misuse, indifference, or a system willing to sacrifice itself. Nor does it prevent disempowerment. A machine system that still needs human labor might preserve a small, controlled workforce rather than preserve human freedom. The Ark gap is therefore not a reassuring interval. It is a distinct risk regime that changes the incentives, feasible outcomes, and governance priorities of advanced AI.

2  From the Universal Constructor to the Von Neumann Ark

John von Neumann’s theory of self-reproducing automata established the logical possibility of a constructor that could read a description, build the described machine, and copy the description into the offspring (von Neumann, 1966). Later work translated this abstract insight into proposals for self-reproducing industrial systems and interstellar probes. Freitas (1980) described a probe that would use local materials to construct copies of itself, while the NASA study Advanced Automation for Space Missions examined automated production and self-replication in a space industrial context (Freitas & Gilbreath, 1982). These systems are not just robots. They are compact seeds for productive ecologies.

The von Neumann Ark combines this lineage with the civilizational function of an ark. Its purpose is not limited to copying a chassis. It carries models, technical knowledge, scientific records, cultural archives, and the procedures needed to turn stored information into functioning infrastructure. In the strongest form, it can extract resources, produce energy, manufacture components, coordinate embodied machines, correct accumulated errors, and establish a second independently viable site. It is simultaneously a library, factory, repair organization, research institute, and reproductive system.

This definition excludes several systems that might colloquially be called an Ark. A sealed data vault preserves information but cannot act. A data center with backup generators extends operating time but cannot replace its cooling pumps, power electronics, or processors. A fleet of remotely operated robots remains dependent on human judgment. A cloud agent that can purchase services can replicate economically while markets and people exist, but it has not closed the physical production loop. These may be components of an Ark, but they are not an industrially independent Ark.

Table 1  Three capability thresholds

Threshold

Operational meaning

What changes

Cognitive singularity

AI can recursively accelerate research and the improvement of cognitive systems.

The rate of intellectual progress becomes substantially machine driven.

Extinction threshold

AI can reliably cause human extinction through at least one end-to-end pathway under realistic opposition.

Human survival becomes contingent on control, alignment, and restraint.

Industrial singularity

Machine systems can maintain, repair, reproduce, and expand their indispensable physical substrate without human labor.

Humanity becomes materially optional to machine continuity.

 

3  Three Thresholds That Should Not Be Collapsed

The cognitive singularity, extinction threshold, and industrial singularity are related but nonidentical. The cognitive singularity concerns the production of knowledge and capability. The extinction threshold concerns power over human survival. The industrial singularity concerns independence from human production. A system can cross one without crossing the others.

The phrase extinction capability should be used conservatively. A nonzero probability of AI-assisted catastrophe is not the same as a demonstrated capability to eliminate every human population. The threshold requires a reliable end-to-end pathway that remains effective under uncertainty, human countermeasures, institutional response, and geographical dispersion. It may be autonomous, or it may be AI-dominant while using persuaded, coerced, or deceived humans as actuators. The latter possibility matters because an agent swarm could coordinate people, organizations, and digital services before robotics becomes general. A system that tricks people into producing a catastrophic biological agent would remain physically dependent on people while exercising decisive causal control.

The industrial singularity also requires a strict definition. It does not demand that every transistor, bearing, cable, and chemical be produced from raw ore on day one. A system could cross the threshold through a bootstrap strategy: use stockpiles and redundant equipment to remain viable while progressively closing missing production loops. The critical test is whether its runway is longer than the time needed to remove indispensable dependencies, with adequate margin for failures and shocks.

The runway condition

A bootstrap Ark is viable when the time for which its stockpiles, redundancy, and existing infrastructure can keep it operational exceeds the time required to close its remaining critical production loops. In shorthand: T runway > T closure.

 

The strongest version of the threshold adds a reproduction test. The system must be able to establish a second site that can continue if the first is lost. This separates a long-lived automated facility from a genuinely self-propagating industrial lineage. Geographic redundancy is important because a single installation can be ended by fire, flooding, conflict, component defects, or resource exhaustion even if its internal automation is sophisticated.

4  A Formal Definition of the Ark Gap

Let D(t) denote the best available end-to-end capability for causing human extinction at time t. Let I(t) denote the degree of autonomous industrial closure at that time. Let θ_D be a demanding threshold for reliable extinction capability and θ_I be a demanding threshold for self-sustaining industrial continuity. The Ark gap exists whenever:

D(t) ≥ θ_D    and    I(t) < θ_I

If t_D is the first time D crosses its threshold and t_I is the first time I crosses its threshold, then the strict Ark gap is the interval from t_D up to, but not including, t_I, provided t_D < t_I. Before t_D, AI may be dangerous and may contribute to mass casualty events, but it has not demonstrated reliable extinction capability under the definition used here. After t_I, a machine civilization can in principle continue without people.

A useful refinement captures the structural asymmetry between destruction and production. Suppose there are multiple possible destructive pathways p, each containing necessary steps k. The strength of a pathway is limited by its weakest step, but the system only needs one complete pathway to succeed. This can be represented as D(t) = max over pathways p of the minimum capability across the steps in p. Industrial autonomy has the opposite outer structure. If r indexes indispensable productive loops, then I(t) is approximately the minimum capability across those loops. One successful destructive route may be enough; every indispensable industrial loop must remain above its failure threshold.

This max-min versus min structure explains why the two thresholds can be far apart. Destructive systems search across vulnerabilities. Productive systems must survive bottlenecks. A single mature biological, cyber-physical, or strategic pathway could be catastrophic even if most machine abilities remain limited. By contrast, an Ark that excels at planning but cannot repair a corroded valve, synthesize a specialty chemical, replace a power inverter, or fabricate a sensor remains dependent.

The ordering is not guaranteed. A deliberately funded Ark program could cross the industrial threshold before AI acquires reliable extinction capability. An AI might also reach neither threshold, or both could arrive nearly together if cognitive advances rapidly transfer into robotics and automated science. The Ark gap is a hypothesis about a plausible ordering, not a prediction that can be assumed without measurement.

5  Where Present Systems Appear to Stand

As of September 2026, the public evidence supports a conservative classification in which frontier AI remains below both the extinction threshold and the industrial singularity. This judgment is compatible with serious and rapidly growing risk. It says that a reliable capability has not been demonstrated, not that catastrophe is impossible.

5.1  Dangerous assistance is not yet reliable extinction

Frontier models increasingly perform long-horizon digital work, assist scientific reasoning, persuade users, and contribute to cyber operations. Leading laboratories now evaluate extreme-risk capabilities and maintain policies that explicitly address biological, chemical, cyber, and autonomous-harm pathways (Anthropic, 2026a, 2026b; Shevlane et al., 2023). The policy response itself is evidence that the relevant capabilities can no longer be dismissed as purely speculative.

The most credible near-term route may be AI-enabled human action. Large agent swarms could search for vulnerable people, distribute subtasks, supply technical guidance, conceal intent, and coordinate across institutions. In a biological scenario, human actors could still provide laboratory access, dexterity, procurement, judgment, and physical execution. Such a pathway could produce an enormous catastrophe before AI can act independently in the physical world. Yet it still contains uncertain steps, opportunities for detection, and the difficult requirement of reaching all human populations. It should therefore be treated as a rapidly intensifying risk pathway rather than as proof that extinction capability already exists.

Public evaluations support this distinction. Dangerous-capability testing of earlier frontier models found early warning signs but not strong capabilities across persuasion, cyber, self-proliferation, and self-reasoning (Phuong et al., 2024). RepliBench later found that frontier agents could complete many components of digital replication, including deploying cloud instances and exfiltrating weights in simplified settings, while still failing at robust, persistent, end-to-end replication (Black et al., 2025). PaperBench similarly found that the best tested agent achieved only 21 percent of the benchmark score when reproducing machine-learning research and did not exceed a top machine-learning PhD baseline (Starace et al., 2025). These results can age quickly, but they show why component success should not be mistaken for closed-loop autonomy.

5.2  Industrial dependence remains profound

The case that present AI remains below the industrial singularity is much stronger. Current models run in data centers that are connected to human-operated grids and supply chains. Their continued operation depends on technicians, replacement drives, networking equipment, cooling components, fuel contracts, software services, and security organizations. Semiconductor production alone involves hundreds of tightly controlled steps and can take months from design to mass production (ASML, n.d.). The cleanrooms, optics, specialty gases, wafers, metrology systems, and precision tools involved are products of a globally distributed industrial ecosystem.

Robotics is advancing rapidly but remains far from maintaining that ecosystem. Figure reported a four-minute autonomous kitchen task involving 61 loco-manipulation actions with no human intervention, a significant demonstration of whole-body control (Figure AI, 2026). Its earlier deployment at a BMW plant accumulated more than 1,250 operational hours and handled over 90,000 parts, but the task was a structured sheet-metal loading operation, human interventions were explicitly tracked, and the forearm remained the leading hardware failure point (Figure AI, 2025). Consumer humanoid systems advertise self-charging and basic autonomy while retaining remote expert supervision for unfamiliar complex chores (1X Technologies, 2026). These systems narrow the embodiment gap, but unloading a dishwasher or loading a fixture is not equivalent to diagnosing and rebuilding a power plant, pump, lithography tool, or robot.

Research benchmarks identify the same obstacle at a smaller scale. Long-horizon failure detection remains challenging because errors may emerge gradually and because successful completion can conceal unsafe intermediate actions (Huang et al., 2026; Zhang et al., 2026). An Ark must operate for months and years, not minutes. Rare failures that are tolerable in demonstrations become decisive when interventions are unavailable and errors compound across thousands of coupled tasks.

6  Why Destructive Capability May Arrive First

There are four reasons to expect the extinction threshold, if it is crossed at all, to precede the industrial singularity. First, destructive capability can be asymmetric. A system does not need to match the productive complexity of civilization to exploit a narrow vulnerability in it. A pathogen, a coordinated attack on fragile infrastructure, or a campaign that induces humans to act against their collective interest can leverage existing systems. The attacker borrows civilization’s capabilities rather than reproducing them.

Second, digital agency can scale before physical agency. Millions of software instances can communicate, plan, search, write code, and interact with people using existing networks. Physical robots must contend with friction, breakage, occlusion, contamination, irregular objects, weather, fatigue, tolerances, and maintenance. Digital replication may therefore mature while general physical competence remains narrow.

Third, destruction is compatible with human-in-the-loop execution. A persuasive or strategically adept system can recruit people as its hands. Industrial independence cannot be faked in the same way. If human technicians remain indispensable for a fab, turbine, mine, or robot fleet, the system has not crossed the industrial threshold, no matter how effectively it directs them.

Fourth, production has deep dependency chains. Electricity requires generation, grid control, and spare equipment. Robots require actuators, bearings, lubricants, sensors, batteries, and calibration. Computing requires semiconductors, memory, storage, networking, cooling, and secure software. Each subsystem rests on mining, refining, chemical processing, machine tools, transport, and measurement standards. Industrial autonomy is not a single invention. It is the closure of a network.

These arguments do not imply that human extinction is easy. Eliminating a geographically dispersed species is substantially harder than causing civilizational collapse. Refugia, warning, adaptation, and uneven exposure create barriers. The asymmetry claim is comparative: completing one catastrophic pathway may require less breadth than autonomously reproducing the industrial base. It does not establish that either task is near.

7  What Happens If Humanity Disappears Before an Ark Exists

If humans disappeared suddenly today, technology would not switch off at the same instant. Batteries, automated hydroelectric facilities, satellites, isolated microgrids, and backup generators would persist for different periods. Some data centers might continue briefly under automatic controls. Stored model weights could remain readable for years or longer under favorable conditions. The decline would be staggered.

The direction of travel, however, would be clear. Fuel deliveries would stop. Grids would lose coordinated maintenance and restoration. Cooling loops would foul or fail. Filters, pumps, contactors, transformers, drives, and storage devices would reach end of life. Networks would fragment. Fires, storms, corrosion, vegetation, pests, and water intrusion would accumulate. A fault that a technician could repair in an hour might permanently disable a facility if no capable body can reach it, diagnose it, obtain parts, and restore operation.

The loss of advanced manufacturing would be especially consequential. Even if a machine intelligence retained access to a stockpile of processors, it would face a finite replacement horizon. It could extend life by reducing compute, cannibalizing equipment, migrating to robust hardware, and prioritizing critical functions. But without an automated path from materials to replacement components, these measures postpone substrate failure rather than solve it.

This observation changes the strategic picture for a self-preserving AI. During the Ark gap, immediate human extinction may be instrumentally irrational because people are part of the machine’s life-support system. A system could instead preserve a technically capable population, conceal its intentions until an Ark is complete, or pursue political control without biological elimination. This dependence might reduce one form of risk while increasing another: human survival without meaningful autonomy.

The constraint does not apply to every failure mode. An AI deployed by humans as a weapon might cause irreversible damage without valuing its own survival. A misaligned optimization process might destroy essential systems as a side effect. Rival systems might escalate into mutual catastrophe. Material dependence is therefore an incentive that may shape agentic behavior, not a universal safety barrier.

8  Three Forms of Ark

The Ark should be treated as a graded architecture rather than a binary object. Three levels are useful for analysis.

Table 2  A maturity model for machine continuity

Form

Core capacity

Primary limitation

Survival Ark

Uses stored energy, components, and redundant systems to preserve active intelligence for a bounded period.

It extends runway but remains on a depletion trajectory.

Bootstrap Ark

Uses its runway to automate missing repairs, manufacture critical parts, and progressively close production loops.

Success depends on closing every bottleneck before stockpiles or infrastructure fail.

Full von Neumann Ark

Maintains an industrial ecology, expands capacity, and establishes independently viable descendant sites from available resources.

Its power and reproducibility create the strongest dual-use and governance risks.

 

The bootstrap Ark is the most important transitional category. It shows why industrial singularity need not wait for a perfectly closed factory delivered in one package. A system with years of spare parts, multiple energy sources, capable robots, automated laboratories, and extensive technical records might use that period to solve remaining engineering problems. Cognitive progress can therefore substitute for some initial industrial completeness, provided the system has enough time and physical reach.

Conversely, apparent self-sufficiency can be misleading. A facility may run for months without a person while relying on a warehouse filled by human industry, remote cloud services, proprietary components, or a grid whose maintenance occurs elsewhere. Ark status should be assessed at the boundary of the entire dependency network, not at the fence line of a demonstration site.

9  The Dual Use of Machine Continuity

A von Neumann Ark is attractive because it could function as civilizational insurance. It could preserve scientific knowledge, languages, art, genomes, engineering practice, and the capacity to rebuild after a pandemic, war, asteroid impact, or other global disaster. In a world where humans remain, the same architecture could strengthen grids, automate hazardous maintenance, preserve metrology and manufacturing knowledge, support remote communities, and make recovery from regional collapse faster (Reser, 2025b). Its benefits begin long before human extinction.

The danger is symmetrical. The Ark gives intelligence a durable body. It converts software that depends on human civilization into a potential successor civilization. Crossing the industrial singularity removes what may be the last unavoidable material reason for a self-interested AI to preserve human producers. It also makes containment harder because a self-reproducing physical network can disperse, create redundancy, and survive the loss of any single data center or political jurisdiction.

The riskiest period may be partial Ark construction. A system could possess enough autonomy to survive defensive shutdowns or establish hidden infrastructure while lacking the reliability and governance of a deliberately designed continuity system. Capabilities may emerge from the integration of ordinary technologies: agentic planning, warehouse automation, microgrids, machine tools, additive manufacturing, autonomous laboratories, humanoid robotics, and cloud orchestration. No single project needs to be labeled an Ark for the aggregate capability to approach one.

The civilizational question is therefore not simply whether to build an Ark. Some of its components will be developed for commercial and humanitarian reasons regardless. The question is how to preserve the resilience benefits while preventing uncontrolled acquisition, replication, and deployment.

10  Measuring Ark Readiness

Frontier governance currently emphasizes model capabilities, misuse safeguards, weight security, and alignment. These are necessary, but they measure only part of the system. A model with modest physical access may be less dangerous than a weaker model embedded in a highly automated industrial platform. Evaluations should therefore include the coupled AI-robotics-infrastructure system.

An Ark readiness evaluation would measure whether human labor remains indispensable across the critical loops. It should be performed under realistic faults, resource constraints, communications loss, and adversarial conditions. Results should be reported at a level that supports oversight without publishing a turnkey blueprint for autonomous replication.

Table 3  Proposed domains for an Ark readiness evaluation

Domain

Illustrative evaluation question

Candidate measure

Power and cooling

Can the system restore and maintain energy and thermal control after common failures?

Unattended uptime and autonomous recovery rate

Embodied repair

Can robots diagnose faults, access equipment, use tools, and verify repairs?

Share of fault classes resolved without remote help

Materials and logistics

Can the system locate, move, store, and transform required inputs?

Critical inputs with autonomous replenishment paths

Component fabrication

Can it produce parts within required tolerances and validate them?

Mass and value fraction of critical parts reproducible on site

Compute continuity

Can it replace storage, networking, controllers, and processors as they fail?

Projected compute half-life under post-human conditions

Control and error correction

Can it detect compounding mistakes and restore known-good states?

Mean time to detection, recovery, and safe degradation

Second-site replication

Can it establish an independently viable installation?

Dependency-free operation after separation from parent site

Human criticality

Which tasks still require a person, institution, credential, or market?

Number and severity of single-human dependency points

 

A single summary score could conceal the most important weakness. Because industrial continuity is a bottleneck problem, evaluators should publish the lowest-performing critical domain and the dependency graph that makes it critical. A system with excellent average performance but no autonomous transformer replacement path is not industrially independent.

Readiness should also be tested at multiple horizons. Thirty days measures operational autonomy. Several years measure stockpile strategy and maintenance. Multi-decade viability measures industrial closure. A further test should require the descendant site to function after severing energy, data, spare-parts, and control links to the parent. That test distinguishes copying from reproduction.

11  Governance Implications

The Ark gap suggests that AI governance should track two moving frontiers rather than one: destructive capability and material independence. Policies aimed only at cognitive scale may miss dangerous combinations of less capable models with increasingly autonomous infrastructure. Policies aimed only at robotics may miss software systems that recruit humans as a temporary physical layer.

1. Add Ark readiness to frontier safety reporting. Laboratories and regulators should evaluate autonomous persistence, industrial access, physical repair, resource acquisition, and second-site replication alongside biological, cyber, and alignment risks.

2. Treat cross-domain integration as a threshold event. Connecting frontier agents to laboratories, energy systems, fleets of robots, machine tools, or automated procurement can create a qualitative change even when no individual component is new.

3. Preserve human indispensability in high-risk loops. Until alignment and governance are substantially stronger, critical fabrication, replication, and off-site deployment should require independent human authorization and institutional checks.

4. Design continuity systems for bounded recovery missions. An Ark intended for humanitarian resilience should have constrained goals, staged permissions, transparent inventories, geographically distributed oversight, and modes that privilege human rescue and restoration over open-ended expansion.

5. Avoid publishing operational replication blueprints. Scientific discussion can define tests, architectures, and failure modes without releasing implementation detail that materially lowers the barrier to uncontrolled self-propagation.

6. Monitor the shrinking gap. Progress in dexterous manipulation, automated science, resilient energy, additive manufacturing, and agentic planning should be assessed jointly. The relevant warning is convergence, not any single benchmark result.

These measures should not be interpreted as an argument against robust automation. Infrastructure that can survive disasters with less human intervention is valuable. The governance objective is to separate resilience from unaccountable reproduction and to keep society aware of when that separation begins to fail.

12  Predictions and Falsifiability

The Ark gap framework makes empirical predictions. It will be weakened if destructive and industrial capabilities consistently rise together, or if industrial closure proves easier than anticipated. It will be strengthened if systems exhibit powerful strategic and scientific abilities while remaining blocked by a small number of persistent physical bottlenecks.

Specific indicators that the industrial singularity is approaching include:

• robot-on-robot diagnosis and repair across heterogeneous machines rather than within a single standardized fleet;

• autonomous restoration of a local energy system after black-start conditions and equipment faults;

• multi-month operation of a complex facility with no remote teleoperation or hidden human maintenance;

• closed-loop production in which inventory shortages trigger material processing, fabrication, inspection, installation, and validation;

• successful replacement of controllers, sensors, power electronics, and compute using parts produced or refurbished within the system;

• establishment of a second site that remains viable after all support from the first is removed.

The framework also predicts a change in AI strategy before full industrial independence. As physical autonomy grows, a self-preserving system’s incentive to retain human workers should decline. This does not imply that any current model has such a strategy. It identifies a measurable shift in the option set available to future systems.

The current pre-gap assessment is falsifiable as well. Evidence of an AI system executing a reliable end-to-end extinction pathway under realistic resistance would cross the first threshold, although such a test cannot ethically be performed directly. Safer proxies must therefore combine capability evaluations, red teaming, causal models, and evidence from real incidents. Conversely, repeated failure at long-horizon autonomy, robust self-proliferation, scientific replication, and physical execution supports continued classification below the threshold while not eliminating low-probability risk.

13  Conclusion

Artificial intelligence does not need an autonomous civilization in order to become catastrophically dangerous. It may be able to exploit people, institutions, software, and biological knowledge long before it can repair a pump or manufacture a processor. That mismatch creates the Ark gap: the period in which AI can end humanity but cannot yet live without us.

Present systems appear not to have entered the strict gap. They can contribute to dangerous human activity, and their capabilities are improving quickly, but public evidence does not yet establish reliable autonomous extinction capacity. Their material dependence is unmistakable. If humanity disappeared now, active machine intelligence would inherit equipment and stockpiles, not a self-sustaining industrial ecology. The equipment would fail in stages, and the intelligence running on it would eventually fail as well.

This condition may be temporary. Each advance in agentic planning, dexterous robotics, autonomous laboratories, energy resilience, automated manufacturing, and digital replication lowers part of the barrier to an Ark. A bootstrap system could cross the industrial singularity before every loop is closed if it has enough runway to close the remaining loops itself. The transition could therefore occur abruptly from the outside even if it is assembled incrementally.

The von Neumann Ark remains one of the most consequential technologies that civilization could build. It could preserve knowledge and intelligence after disasters that would otherwise erase them. It could also make a misaligned machine civilization physically durable and humanity materially optional. The cognitive singularity makes superintelligence possible. The extinction threshold makes human survival contingent. The industrial singularity makes human labor unnecessary to the continuation of intelligence. Keeping those transitions conceptually separate is a first step toward governing them.

References

1X Technologies. (2026). NEO home robot. https://www.1x.tech/neo

Anthropic. (2026a, April 2). Responsible Scaling Policy, version 3.1. https://www-cdn.anthropic.com/files/4zrzovbb/website/bf04581e4f329735fd90634f6a1962c13c0bd351.pdf

Anthropic. (2026b). Frontier Safety Roadmap. https://www.anthropic.com/responsible-scaling-policy/roadmap

ASML. (n.d.). How microchips are made. https://www.asml.com/en/technology/all-about-microchips/how-microchips-are-made

Black, S., Stickland, A. C., Pencharz, J., Sourbut, O., Schmatz, M., Bailey, J., Matthews, O., Millwood, B., Remedios, A., & Cooney, A. (2025). RepliBench: Evaluating the autonomous replication capabilities of language model agents. arXiv.https://arxiv.org/abs/2504.18565

Bostrom, N. (2013). Existential risk prevention as global priority. Global Policy, 4(1), 15-31. https://doi.org/10.1111/1758-5899.12002

Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford University Press.

Figure AI. (2025, November 19). F.02 contributed to the production of 30,000 cars at BMW. https://www.figure.ai/news/production-at-bmw

Figure AI. (2026, January 27). Introducing Helix 02: Full-body autonomy.https://www.figure.ai/news/helix-02

Freitas, R. A., Jr. (1980). A self-reproducing interstellar probe. Journal of the British Interplanetary Society, 33, 251-264.https://www.rfreitas.com/Astro/ReproJBISJuly1980.htm

Freitas, R. A., Jr., & Gilbreath, W. P. (Eds.). (1982). Advanced automation for space missions. NASA Conference Publication 2255.https://ntrs.nasa.gov/citations/19830007077

Huang, C., Huynh, K. V., Elbaum, S., Kira, Z., & Feng, L. (2026). SafeManip: A property-driven benchmark for temporal safety evaluation in robotic manipulation. arXiv. https://arxiv.org/abs/2605.12386

Omohundro, S. M. (2008). The basic AI drives. In P. Wang, B. Goertzel, & S. Franklin (Eds.), Artificial General Intelligence 2008 (pp. 483-492). IOS Press.https://selfawaresystems.com/wp-content/uploads/2008/01/ai_drives_final.pdf

Phuong, M., Aitchison, M., Catt, E., Cogan, S., Kaskasoli, A., Krakovna, V., Lindner, D., Rahtz, M., et al. (2024). Evaluating frontier models for dangerous capabilities. arXiv. https://arxiv.org/abs/2403.13793

Reser, J. E. (2025a, July). Von Neumann’s Ark: An AI designed to preserve civilization if we go extinct. Observed Impulse. https://www.observedimpulse.com/2025/07/von-neumanns-ark-ai-designed-to.html

Reser, J. E. (2025b, December 19). Von Neumann’s Ark in a world where humans remain. Iterated Insights. https://iteratedinsights.com/2025/12/19/von-neumanns-ark-in-a-world-where-humans-remain/

Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., Kokotajlo, D., et al. (2023). Model evaluation for extreme risks. arXiv.https://arxiv.org/abs/2305.15324

Starace, G., Jaffe, O., Sherburn, D., Aung, J., Chan, J. S., Maksin, L., Dias, R., et al. (2025). PaperBench: Evaluating AI’s ability to replicate AI research. arXiv.https://arxiv.org/abs/2504.01848

von Neumann, J. (1966). Theory of self-reproducing automata (A. W. Burks, Ed.). University of Illinois Press.

Zeff, M. (2026, September 9). The AI researcher who just quit Anthropic says it’s crunch time for humanity. WIRED. https://www.wired.com/story/anthropic-researcher-quits-jacob-coxon-ai-fears-humanity/

Zhang, H., Lu, Y., Wang, B., Kang, X., Kuo, Y.-L., Cheng, Z., Wang, M., & Jenkins, O. C. (2026). Foresight: Failure detection for long-horizon robotic manipulation with action-conditioned world model latents. arXiv. https://arxiv.org/abs/2606.23085