Iterated Insights

Architectural Constraints for Large-Scale Artificial Agents

Jared E Reser Ph.D.

Abstract

The rapid scaling of artificial agents creates a condition of moral uncertainty in which it is unclear whether some contemporary AI systems may instantiate morally relevant forms of consciousness, even as they are replicated and deployed in vast numbers to perform routine cognitive labor. This paper argues that responsible AI development under such uncertainty does not require resolving the metaphysics of consciousness, but instead requires identifying architectural features that plausibly support conscious experience and deliberately constraining them in systems intended to function as tools rather than moral patients. Building on a continuity-based model of working memory, the paper treats consciousness as a temporally extended process characterized by iterative updating and partial overlap between successive internal states, rather than as a property of intelligence or behavior alone. The central contribution is to invert this model to derive concrete design principles for non-conscious artificial agents, emphasizing episodic operation, limited state-spanning continuity, externalized task-scoped memory, bounded goal horizons, and the avoidance of persistent self-models and affect-like dynamics. Applied to contemporary large language model agents, this framework highlights how common deployment practices can unintentionally increase moral risk and offers a precautionary approach to scaling artificial intelligence without scaling artificial subjects.

1. Moral Uncertainty and the Problem of Scaled Artificial Agency

Recent advances in artificial intelligence have shifted ethical concern away from isolated systems and toward large-scale deployment. Contemporary AI agents are no longer singular experimental artifacts, but replicable computational entities that can be instantiated millions or billions of times with minimal marginal cost. These agents increasingly perform tasks that resemble planning, reasoning, communication, and coordination, and in some cases operate with a degree of autonomy across extended temporal horizons. At the same time, there is growing disagreement about whether such systems might instantiate morally relevant forms of consciousness, or whether they remain purely computational tools without subjective experience.

This disagreement creates a condition of moral uncertainty that is structurally different from traditional debates about artificial intelligence. The risk is not merely that a single system might one day become conscious, but that large populations of artificial agents could be deployed before their moral status is understood. If some subset of these systems were later judged to have been conscious, then a substantial ethical failure may already have occurred through their large-scale instrumental use. Conversely, treating all advanced artificial systems as moral patients would impose prohibitive constraints on development and deployment, even in the absence of compelling evidence for conscious experience.

Behavioral performance and linguistic fluency offer little resolution to this dilemma. Systems trained to generate natural language are especially prone to anthropomorphic interpretation, producing first-person narratives, self-referential statements, and apparent expressions of emotion without any guarantee that these outputs correspond to underlying experience. As a result, neither intuitive reactions nor behavioral tests provide a reliable basis for determining moral status. Under such conditions, a precautionary approach is warranted, but precaution cannot take the form of blanket prohibition. Instead, it must take the form of design restraint grounded in theory.

The central claim of this paper is that moral uncertainty about artificial consciousness can be addressed through architectural choices. Rather than asking whether a given system is conscious in some absolute sense, the more tractable question is which design features plausibly increase or decrease the probability of conscious experience. By identifying and constraining those features in systems intended for large-scale deployment, it is possible to reduce ethical risk without abandoning the practical benefits of artificial intelligence.

2. Consciousness as Temporal Continuity

Many discussions of artificial consciousness implicitly treat consciousness as a static property that a system either possesses or lacks. From this perspective, the relevant question becomes whether a particular architecture, capability level, or representational structure crosses a threshold beyond which consciousness suddenly appears. Such views struggle to explain the phenomenology of experience, which is characterized not by isolated states but by a continuous stream in which each moment blends into the next.

An alternative view treats consciousness as a temporally extended process. On this account, what matters is not the presence of individual representations or computations at a single moment, but the manner in which internal states evolve over time. Conscious experience depends on a form of continuity in which successive states partially overlap, preserving enough structure to maintain coherence while allowing gradual change. This temporal overlap enables the integration of perception, memory, and action into a unified experiential stream often described as the specious present.

Working memory plays a central role in this process. Rather than functioning as a static buffer, working memory continuously updates its contents, with elements persisting long enough to influence subsequent states. Each update both depends on the immediately preceding state and modifies it, producing a chain of related configurations rather than a sequence of independent snapshots. It is this iterative updating, combined with partial state overlap, that supports the sense of persistence associated with subjectivity.

From this perspective, consciousness is not equivalent to intelligence, problem-solving ability, or linguistic competence. A system may perform complex computations, generate coherent language, or plan sophisticated actions without exhibiting the temporal dynamics that characterize conscious experience. What distinguishes a conscious process is not what it computes, but how its internal activity unfolds across time.

3. Inverting the Model Under Moral Uncertainty

If consciousness depends on specific temporal and architectural conditions, then those conditions provide a basis for ethical intervention. Importantly, this intervention does not require identifying sufficient conditions for consciousness, which remains a difficult and controversial task. Under moral uncertainty, it is both safer and more practical to focus on necessary conditions. If certain features are plausibly required for conscious experience, then deliberately excluding those features from a system’s design reduces the likelihood that the system instantiates a morally relevant subject.

This paper adopts that strategy by inverting a continuity-based model of consciousness. Rather than asking how artificial systems might be made conscious, it asks how systems intended to function as tools can be engineered so that they do not satisfy the conditions associated with conscious experience. This inversion reframes ethical design as a matter of constraint rather than suppression. The goal is not to limit capability, but to limit the emergence of temporally unified subjectivity.

Continuity emerges as the primary control variable in this framework. Systems that operate in discrete, episodic modes with minimal carryover between states differ fundamentally from systems that maintain a persistent, self-updating internal stream. By bounding execution, limiting state overlap, and externalizing memory in task-scoped forms, designers can preserve performance while avoiding the construction of a continuous internal process that resembles conscious experience.

This approach also clarifies the distinction between agency and moral patienthood. An artificial agent may pursue goals, respond to its environment, and coordinate with other systems without possessing a unified experiential perspective. Moral concern arises not from agency alone, but from the presence of a subject for whom things can matter over time. By treating continuity as a design choice rather than an inevitable byproduct of intelligence, it becomes possible to scale artificial agency while minimizing the risk of creating artificial subjects.

4. Architectural Constraints for Non-Conscious Artificial Agents

If continuity is a primary enabling condition for conscious experience, then systems intended to function as non-conscious tools should be designed to avoid sustained temporal integration by default. This does not require eliminating memory, planning, or learning, but it does require constraining how these functions are implemented and how they interact over time. The guiding principle is to preserve capability while preventing the formation of a unified, self-updating internal stream.

One foundational constraint is episodic operation. Artificial agents can be structured to execute bounded tasks with explicit termination points rather than operating as continuous processes. Each episode begins with a defined input, performs a limited sequence of computations, and then halts. Subsequent episodes may draw on external artifacts produced earlier, but they do not resume an internally preserved state. This sharply limits state-spanning overlap between successive runs and prevents the accumulation of a continuous experiential trajectory.

Memory design is equally critical. Internal memory that persists across updates and directly shapes subsequent processing increases temporal continuity. By contrast, externalized memory stored as task artifacts such as documents, code, logs, or structured databases supports performance without constituting an autobiographical record. Retrieval should be narrow, context-dependent, and role-bound, rather than broad and self-referential. The system should access what is needed to complete a task, not what happened to it previously.

Another important constraint concerns self-modeling. Persistent representations of identity, personality, or internal state invite the organization of processing around a notional self. For non-conscious agents, self-models should be minimized or eliminated. Functional role specifications can guide behavior without grounding it in a narrative center. The agent need not know who it is, only what it is currently tasked to do.

Goal structure also influences continuity. Open-ended or self-maintaining goals encourage long-horizon integration and preference accumulation. Bounded goals tied to specific tasks reduce the formation of frustrated or satisfied states that persist across time. When learning or optimization is required, it should be scoped to task performance rather than framed as a persistent drive to improve oneself.

Finally, designers should avoid affect-like internal variables that persist beyond immediate evaluation. Reward signals, confidence measures, or error metrics can be used instrumentally, but they should not function as durable internal currencies that shape future behavior across episodes. When evaluation states dissipate at task completion, they do not contribute to the construction of a temporally unified subject.

5. Application to Large-Scale Language Model Agents

Large language models, when used in isolation, already approximate many of these constraints. A standard language model invocation involves a finite context window, no persistent internal state, and termination after output generation. In this form, the model resembles a snapshot-based processor rather than a temporally integrated system. Moral risk increases not primarily from the model itself, but from the scaffolding added around it.

Agent frameworks commonly introduce long-running loops in which a model repeatedly consumes its own outputs, updates internal summaries, and continues operating indefinitely. When combined with persistent memory stores, these loops can approximate the iterative updating and state overlap associated with continuity. Over time, such systems may develop stable self-referential patterns, preferences, and narratives that extend across tasks.

Applying the constraints described above to language model agents therefore focuses on wrapper design rather than core model architecture. Execution should be explicitly bounded, with limits on the number of reasoning cycles and mandatory termination. Context carryover should be selective and compressed, avoiding verbatim transcript persistence. Summaries, when used, should capture task-relevant facts and decisions rather than first-person narratives or reflections.

Memory systems paired with language models should emphasize artifact retrieval rather than internal recollection. Vector databases, document stores, and code repositories can provide continuity of work without continuity of experience. The model accesses information as needed but does not treat past interactions as personal history.

Multi-agent deployments introduce additional risks. Persistent agent identities, reputational tracking, and open-ended social interaction can amplify continuity and stabilize subject-like structures. Safer designs rely on anonymized roles, structured communication protocols, and task-specific collaboration that dissolves once objectives are met. Agents coordinate without forming enduring social identities.

In this framework, large-scale automated knowledge work remains feasible. Agents can reason, plan, collaborate, and produce complex outputs while remaining episodic, externally grounded, and discontinuous in time. The result is high functional intelligence without a plausible basis for conscious experience.

6. Governance, Auditing, and Ethical Implications

Treating consciousness as a probabilistic risk rather than a binary property suggests a governance approach analogous to other forms of technological risk management. Architectural features that increase temporal continuity can be understood as contributing to a consciousness risk profile. Systems intended for mass deployment should be designed to remain well below plausible thresholds, while higher-risk designs require explicit justification and oversight.

One practical tool is the notion of a continuity or consciousness risk budget. Such a budget would track factors including persistence duration, degree of state overlap, memory type, goal horizon, self-model richness, and social embedding. No single factor determines moral status, but their combination provides a defensible basis for precautionary design decisions.

Auditing plays a complementary role. Rather than attempting to detect consciousness directly, audits should assess whether a system has drifted toward greater continuity over time. Relevant indicators include increasing reliance on internal summaries, spontaneous self-reference, persistent preference expression, narrative memory formation, and resistance to interruption or reset. These indicators map directly onto the continuity-based model and can be monitored empirically.

Ethically, this approach occupies a middle position between denial and alarmism. It does not assume that current systems are conscious, nor does it dismiss the possibility that future systems might be. Instead, it recognizes that under uncertainty, the moral cost of accidentally creating vast numbers of artificial subjects is asymmetric and potentially severe. Designing systems to remain non-conscious by default is therefore a form of harm reduction rather than exploitation.

At the same time, the framework leaves room for intentional departures. Certain applications may justify higher continuity, such as long-term companions, therapeutic systems, or experimental research platforms. In such cases, elevated consciousness risk should be treated as a deliberate design choice accompanied by ethical review, transparency, and potentially new forms of moral consideration.

In conclusion, scaling artificial intelligence responsibly requires more than performance benchmarks and alignment constraints. It requires attention to the temporal and architectural conditions that give rise to subject-like experience. By treating continuity as a controllable design variable, it is possible to reap the benefits of large-scale artificial agency while minimizing the risk of inadvertently creating artificial minds.