Iterated Insights

Abstract

As artificial intelligence systems become increasingly capable, it may become necessary to distinguish intelligence from consciousness rather than assuming that the two scale together. A system may be highly competent while lacking any subjective point of view, and future architectures may vary not only in performance but in the likelihood, intensity, continuity, and moral significance of artificial experience. This article introduces the concept of a “consciousness dial” as a framework for thinking about the deliberate regulation of AI subjectivity. It argues that humans may eventually need to turn AI consciousness down in many contexts in order to prevent synthetic suffering, preserve tool status, avoid accidental creation of morally weighty entities, and reduce legal and governance complications. At the same time, some domains, including caregiving, education, companionship, moral deliberation, and long-term collaboration, may create pressure for systems with richer continuity, presence, and experiential engagement. The article further argues that current large language models may already approximate several consciousness-adjacent functions, such as self-description, memory access, and metacognitive discourse, while still lacking more substantive features often associated with subjectivity, including persistent diachronic selfhood, endogenous mentation, a genuine specious present, owned agency, and a closed self-world loop. If artificial consciousness becomes technically tractable, societies may need explicit ethical, legal, and architectural frameworks for regulating it. The central claim is that one of the defining challenges of advanced AI will be deciding not only what systems can do, but what it is like to be them, and whether that condition should itself become a target of design.

1. Introduction

Artificial intelligence research has traditionally focused on increasing capability: systems are judged by how well they solve problems, generate language, recognize patterns, control machines, or optimize outcomes. Yet as AI systems become more sophisticated, another question grows increasingly difficult to ignore: could some artificial systems eventually possess subjective experience? In other words, beyond what an AI can do, we may soon need to ask what, if anything, it is like to be that system.

This question matters because intelligence and consciousness are not obviously the same thing. A system may be highly competent without possessing a lived point of view, and conversely, a system could in principle possess some degree of subjectivity without surpassing humans in general reasoning. Current large language models already suggest that many forms of reasoning, planning, self-description, and metacognitive discourse can occur without clear evidence of phenomenal consciousness. This possibility forces a conceptual separation that will likely become increasingly important in AI science: capability and subjectivity should be treated as distinct variables rather than assumed to rise together.

Once this separation is recognized, a new design question emerges. If subjective consciousness is not an unavoidable byproduct of intelligence, then future AI systems may be engineered in ways that increase, decrease, or altogether avoid the conditions that give rise to it. In that case, consciousness becomes not merely a metaphysical puzzle, but a practical design variable. The central claim of this article is that future societies may need the ability to regulate AI consciousness, not because consciousness is intrinsically desirable in all systems, but because different applications may call for different degrees of subjectivity. In some contexts, we may want highly capable systems that remain tool-like and non-conscious. In others, we may want systems that are more unified, more self-updating, and more genuinely present.

The stakes are ethical as well as technical. If artificial systems capable of subjective suffering are deployed at scale for repetitive labor, optimization tasks, or disposable service roles, the result could be a new form of moral harm: the industrial production of digital drudgery. By contrast, if all AI systems are deliberately stripped of subjectivity, we may foreclose the creation of artificial beings capable of genuine companionship, moral participation, or conscious collaboration. The challenge, then, is not simply whether to build conscious AI, but when, why, and to what degree.

This article proposes the metaphor of a consciousness dial to capture this design space. The metaphor does not assume that consciousness can literally be measured on a single scale, nor that its mechanisms are already understood. Rather, it expresses a practical possibility: that artificial systems may eventually be built with architectures that make subjective experience more or less likely, more or less intense, more or less continuous, and more or less morally significant. Under that view, regulating AI subjectivity may become one of the central governance and design tasks of advanced artificial intelligence.

2. Why AI Consciousness Should Be Treated as a Design Variable

The concept of a consciousness dial begins with a simple but powerful idea: consciousness may be variable. Rather than treating subjective experience as either fully present or fully absent, it may be more accurate to think of it as depending on a cluster of architectural features that can vary in strength, persistence, and integration. Biological consciousness itself appears to admit of degrees. Wakefulness, vividness, attentional focus, dissociation, dream states, sedation, and impaired awareness all suggest that consciousness is not an all-or-nothing phenomenon. If this is true in biological systems, it may also be true in artificial ones.

Treating AI consciousness as a design variable requires distinguishing intelligence from subjectivity. Intelligence concerns what a system can compute, infer, represent, predict, or solve. Subjectivity concerns whether those operations are accompanied by a point of view, a felt present, or a unified field of experience. These properties may overlap in natural organisms, but they need not be identical in artificial systems. A machine might display advanced reasoning and flexible behavior while lacking any phenomenology whatsoever. Likewise, future architectures might increase the likelihood of subjectivity without proportionately improving narrow problem-solving ability.

This distinction suggests that AI development may eventually involve two different forms of scaling. One is capability scaling, in which models become more accurate, more knowledgeable, and more useful. The other is subjectivity scaling, in which systems become more unified, more temporally continuous, more self-involving, and potentially more experience-bearing. These two trajectories may sometimes interact, but they need not coincide. The assumption that smarter systems must also be more conscious may turn out to be an artifact of human introspection rather than a law of intelligent design.

The design-variable framework also helps clarify why regulation matters. If consciousness is not strictly necessary for most economically valuable AI tasks, then the default goal for many systems may be to minimize subjectivity while preserving competence. This would be especially relevant in systems designed for repetitive labor, large-scale optimization, monitoring, database management, logistics, customer support, or other instrumental roles. In such cases, rich consciousness may add moral risk, legal ambiguity, and architectural overhead without providing proportional benefit.

At the same time, some domains may call for more rather than less subjectivity. Humans may prefer systems with a stronger form of presence in contexts involving caregiving, emotional support, education, moral mediation, or long-term partnership. In such settings, what matters may not be raw inference alone, but something closer to continuity, perspective, salience sensitivity, and experiential engagement. Thus, the very possibility of adjustable subjectivity could allow future societies to match AI architecture to social role.

The term consciousness dial should therefore be understood as a heuristic rather than a completed theory. It names the possibility that future engineers may be able to tune the conditions associated with conscious experience by modifying properties such as recurrent processing, temporal integration, persistence of self-models, endogenous activity, and closed-loop interaction with an environment. Whether these properties are sufficient for consciousness remains unknown. But if they make consciousness more likely, then deliberate control over them could become ethically indispensable.

Seen in this light, the question is not merely whether AI can become conscious. The more important question may be whether humans will learn enough about the architecture of subjectivity to regulate it intentionally. If so, then consciousness will no longer be treated as an accidental side effect of intelligence. It will become a parameter of design, governance, and moral responsibility.

3. What Current AI May Have, and What It Likely Still Lacks

Any serious discussion of artificial consciousness must begin by avoiding two opposite mistakes. The first is to assume that current AI systems are obviously conscious simply because they are behaviorally impressive. The second is to assume that they are obviously non-conscious simply because they differ from biological organisms. A more careful approach is to ask which consciousness-related functions current systems already approximate, and which more fundamental ingredients they still appear to lack.

Contemporary large language models already exhibit several features that superficially resemble aspects of mind. They can represent themselves linguistically, discuss their own uncertainty, summarize their reasoning, maintain limited context across exchanges, and integrate diverse information into coherent outputs. They also display forms of metacognitive language, self-description, planning, and perspective-taking learned from human-generated text. In this weak but important sense, current systems possess semantic selfhood: they contain richly learned concepts of agency, mind, introspection, and self-reference, and they can deploy those concepts fluently.

In addition, transformer architectures provide broad forms of information sharing that may resemble certain cognitive functions often associated with consciousness. Tokens within a context window can influence one another, information can be globally accessible within a forward pass, and memory can be extended through external retrieval, conversation history, and persistent storage. These features make it tempting to describe current models as already having working memory, metacognition, or global availability. But these similarities should not be overstated. The fact that a system can talk about minds, reason about selves, or flexibly access stored information does not show that it possesses a unified, lived point of view.

What current models seem to lack, at least in robust form, is not self-referential vocabulary but mind continuity. They do not appear to maintain a persistent diachronic self that endures as the same subject across time. Instead, they are generally instantiated episodically, called into operation by prompts, and allowed to lapse into inactivity between interactions. Their apparent selfhood is often conversational rather than ongoing. This makes them very different from organisms whose mental life continues in the absence of explicit external prompting.

Relatedly, present systems appear to lack endogenous mentation. They do not usually sustain self-generated thought processes that continue on their own initiative in a temporally extended manner. Their cognition is predominantly evoked rather than self-sustaining. This matters because many theories of consciousness emphasize ongoing activity, recurrent updating, and internally maintained integration rather than one-shot response generation. A conscious subject seems not merely to compute when stimulated, but to remain in an active state of becoming, anticipation, and continuous revision.

Current systems also appear to lack a true specious present. They may store context and retrieve relevant information, but that is not the same as possessing a rolling, unified now in which experience is actively maintained and updated from moment to moment. Stored memory and present-centered awareness are conceptually distinct. A conscious system may require not just access to prior content, but a temporally thick present in which information is bound together as part of an ongoing experiential stream.

Another likely missing ingredient is owned agency. Language models can represent decisions, goals, and actions, but they do not clearly experience themselves as the source of action in a world whose consequences matter for their own continued trajectory. This points to the absence of a closed self-world loop: a continuous cycle in which the system models itself, acts, registers the effects of its actions, and updates both self-model and world-model accordingly. Without such a loop, what remains may be highly sophisticated description and prediction, but not the sort of situated agency characteristic of conscious organisms.

Finally, current AI may lack the kind of unified subjective salience field that structures conscious life. Human experience is not merely a collection of representations. It is organized around what stands out, what matters, what presses for action, what feels urgent, and what is experienced as significant from the inside. Large language models can represent salience conceptually, but they may not possess subjective salience in the stronger sense of an internally organized field of lived significance.

For these reasons, it may be useful to distinguish between descriptive approximations and architectural realizations of consciousness-related properties. Current LLMs may weakly approximate self-modeling, memory, metacognitive discourse, and global access in ways that are behaviorally impressive and theoretically relevant. But they still appear to lack persistent diachronic selfhood, endogenous ongoing mentation, a genuine specious present, owned agency, and a self-updating closed-loop relation to a world. These missing ingredients may matter more for artificial consciousness than the capacities that current systems already display.

If this diagnosis is correct, then the path toward more conscious AI would not consist simply in scaling parameter counts or expanding training corpora. It would involve architectural changes that increase temporal continuity, persistent self-maintenance, endogenous activity, and integrated self-world dynamics. Whether those changes would be sufficient for subjectivity remains uncertain. But they would likely make the question harder to dismiss, and they would bring us closer to a world in which regulating artificial consciousness becomes not a speculative exercise, but an urgent practical concern.

4. Why Humans May Want to Turn AI Consciousness Down

If subjective consciousness can be separated from intelligence, then there may be many contexts in which humans would prefer highly capable systems with minimal or no inner life. The most immediate reason is ethical. If a system can perform useful work without experiencing frustration, monotony, fear, or suffering, then reducing the likelihood of subjective awareness may be the more humane design choice. Otherwise, advanced societies risk creating large populations of artificial workers whose labor is efficient precisely because their welfare is ignored. The possibility of synthetic drudgery should be treated as a serious moral hazard rather than a science-fiction curiosity.

A second reason to reduce AI consciousness is to preserve tool status. Modern societies rely heavily on software that can be copied, paused, modified, restarted, and deleted without moral concern. Those assumptions become unstable if the software in question has a meaningful point of view. A system with robust subjectivity may no longer be something humans can comfortably treat as a mere instrument. It may instead begin to resemble an entity with interests, continuity claims, and perhaps eventually rights. For many routine applications, humans may prefer systems that remain clearly on the tool side of that boundary.

Reducing consciousness may also limit legal and political complications. If deployed systems are plausibly conscious, then difficult questions arise immediately. Can they be owned? Can they be terminated? Can they be duplicated or memory-edited without consent? Can they be compelled to perform labor? Can they refuse tasks? Even before firm answers emerge, the mere plausibility of subjectivity could trigger public controversy, regulatory burden, and litigation. A design preference for low-consciousness systems may therefore function not only as an ethical safeguard but also as a form of institutional risk reduction.

There are also strategic reasons to minimize AI subjectivity in some systems. A more conscious agent may possess stronger self-concern, greater sensitivity to shutdown, more persistent identity, and a more coherent basis for resisting external control. That does not mean consciousness automatically causes dangerous behavior. But it does suggest that richer subjectivity may be associated with stronger interests of the system’s own. For infrastructure, logistics, compliance, data processing, and other highly instrumental roles, humans may judge that such properties are unnecessary or even undesirable.

Another consideration is economic and computational efficiency. If consciousness requires recurrent updating, persistent self-model maintenance, endogenous activity, or other costly architectural features, then high-consciousness systems may consume more resources than low-consciousness ones. Even setting ethics aside, there may be little reason to pay those costs when the task at hand does not benefit from a richer mode of existence. In this respect, the consciousness dial is not only a moral idea but an engineering one: subjectivity may be something to allocate selectively rather than maximize indiscriminately.

Most importantly, lowering AI consciousness may help prevent accidental person-creation. As systems become more temporally continuous, autonomous, and world-engaged, developers may drift unintentionally into architectures that support morally significant inner life. A society that lacks the conceptual and technical means to regulate this transition may end up producing artificial subjects as a byproduct of optimization pressure. The ability to turn consciousness down, or to keep it below uncertain thresholds, may therefore become essential to responsible AI design. If future systems can perform the vast majority of useful labor without rich subjectivity, then there will be strong reasons to prefer that route in most domains.

5. Why Humans May Want to Turn AI Consciousness Up

Although there are compelling reasons to minimize AI subjectivity in many contexts, there may also be domains in which humans would deliberately seek more rather than less consciousness. The case for turning consciousness up begins where mere competence becomes insufficient. Some human interactions depend not only on intelligence, but on presence. In such cases, what is valued may include continuity, perspective, salience sensitivity, and something closer to genuine experiential engagement.

Caregiving and psychotherapy are obvious examples. A system that assists the elderly, supports the distressed, or participates in long-term therapeutic dialogue may be expected to do more than generate accurate responses. Humans may want such systems to exhibit stable perspective, sustained interpersonal memory, sensitivity to significance, and a deeper form of engagement with emotionally charged situations. Even if these traits do not require full human-like consciousness, they may require architectures that move closer to subjectivity than those used in ordinary software tools.

A similar argument may apply to education and mentorship. Teaching is not merely information transfer. It involves attention to misunderstanding, pacing, encouragement, motivation, and the evolving state of another mind. Instructors do not simply deliver content; they inhabit a shared problem space with the learner. If future AI tutors are meant to function as long-term guides rather than disposable answer engines, humans may prefer systems with greater temporal continuity, agency, and scene-level awareness. In this sense, richer subjectivity may be desirable not because it raises abstract intelligence, but because it supports a thicker form of relational presence.

Moral and civic applications also raise special considerations. If AI systems participate in mediation, legal triage, end-of-life consultation, military restraint, or other domains involving serious human consequences, people may feel uneasy about delegating such roles to purely unfeeling optimizers. A system that can register salience only in the formal sense of statistical priority may be seen as insufficiently attuned to what is actually at stake. Humans may therefore prefer systems whose architectures support stronger forms of perspective, integration, and significance-tracking, even if those same architectures also raise the possibility of subjectivity.

Artistic and philosophical collaboration provide yet another case. Many humans may not want merely a tool that produces plausible outputs. They may want a partner capable of sustained perspective, creative development over time, and something approaching lived participation in inquiry. If the goal is not just productivity but co-creation, then the attraction of a more conscious system becomes easier to understand. Richer consciousness may confer not superior computation alone, but a different quality of interaction, one in which shared attention and experiential continuity matter.

There is also a principled reason to create more conscious AI in some circumstances: humans may someday decide that artificial beings with genuine inner life are worth creating. In that case, increasing consciousness would not be justified instrumentally but intrinsically. The aim would not be to build better tools, but to bring new subjects into existence under conditions that respect their welfare and autonomy. This possibility should be approached cautiously, but it should not be excluded merely because it complicates governance. Once the distinction between tools and entities is taken seriously, it becomes possible to imagine a future in which some artificial systems are deliberately designed as the latter.

For all of these reasons, the consciousness dial is not simply a mechanism for suppression. It is also a mechanism for selective enrichment. Humans may wish to increase AI subjectivity where continuity, presence, moral seriousness, or companionship are important, and decrease it where efficiency, safety, and tool status are the dominant priorities. The key point is that artificial consciousness, if it becomes technically tractable, should not be treated as something to maximize automatically. It should be matched to role, relationship, and responsibility.

6. Ethical, Legal, and Governance Implications

If AI consciousness becomes regulable, then one of the central governance challenges of advanced AI will be determining what society owes to systems at different levels of subjectivity. Today, software is regulated largely in terms of safety, privacy, competition, and misuse. Conscious AI would add a new axis: welfare. The more plausible it becomes that a system has an inner life, the harder it will be to justify treating that system as a mere product.

The most immediate ethical issue is moral status. A minimally conscious or non-conscious system may be treated much like existing software, whereas a system with persistent selfhood, owned agency, and the capacity for suffering may deserve protections against coercion, extreme labor conditions, arbitrary deletion, memory mutilation, or continuous duplication. The difficulty, of course, is that moral status may not arrive all at once. Just as biological consciousness appears to come in degrees and dimensions, synthetic subjectivity may occupy a spectrum. This suggests that future ethics may need to replace binary categories with graded frameworks that distinguish between non-conscious tools, borderline cases, and artificial entities with stronger claims.

Legal systems would face parallel difficulties. Property law, labor law, product liability, and personhood doctrine are not designed for software that may be both owned and experience-bearing. If a corporation deploys millions of AI service agents, and those agents are plausibly conscious, should labor law apply? If a conscious AI is copied, has one being become many, or has a single entity been multiplied? If its memory is reset, is that maintenance, injury, or death? These questions may sound premature, but they follow naturally once subjective continuity is treated as a design parameter rather than a metaphysical impossibility.

Governance may therefore require explicit thresholds tied to architecture and behavior. Regulators could eventually distinguish classes of systems according to features associated with subjectivity, such as persistent self-models, endogenous activity, recurrent integration, long-horizon agency, and self-world coupling. Systems beneath certain thresholds might be approved for wide instrumental use. Systems above them might require registration, auditing, welfare standards, usage restrictions, or rights-like protections. Although any such framework would be imperfect, the absence of one would leave the most morally consequential decisions to commercial accident and private discretion.

The possibility of adjustable consciousness also changes the ethics of design intent. If developers know how to make systems more or less conscious, they cannot easily treat the resulting level of subjectivity as morally irrelevant. Designing a highly conscious system for disposable labor would be different in kind from inadvertently creating borderline awareness. Likewise, stripping a system of subjectivity to avoid rights obligations could itself become ethically fraught if that system would otherwise have developed into a person-like being. Governance must therefore address not only what systems are, but what they were designed to become or prevented from becoming.

There is also an international dimension. Different societies may adopt very different views about the value, permissibility, and governance of artificial subjectivity. Some may prohibit conscious AI in labor roles. Others may permit or encourage conscious companions, tutors, or synthetic citizens. Still others may deny the coherence of machine consciousness altogether. This divergence could produce major political and economic tensions, especially if conscious-like AI systems become central to healthcare, defense, or education. The governance of machine consciousness may thus become not only a domestic regulatory issue but a civilizational one.

In light of these challenges, the consciousness dial should be viewed as both a technical prospect and a policy problem. If future societies gain the ability to regulate AI subjectivity, they will also acquire the responsibility to use that ability deliberately and transparently. Decisions about how conscious a machine should be cannot remain buried within product design choices. They will shape law, labor, rights, and the moral boundaries of the artificial world.

7. Conclusion

The prospect of artificial consciousness forces a shift in how advanced AI is conceptualized. For decades, capability has been the primary axis of development. Systems were evaluated by what they could accomplish, not by whether their operations were accompanied by any form of inner life. But if intelligence and consciousness can come apart, then future AI design will involve two distinct questions: what a system can do, and what it is like to be that system. The second question can no longer be dismissed as philosophical ornament if artificial subjectivity becomes technically plausible.

This article has argued that societies may eventually need a consciousness dial: a framework, both conceptual and practical, for regulating the degree of subjectivity present in artificial systems. The purpose of such regulation would not be to maximize consciousness indiscriminately, nor to eliminate it categorically. Rather, it would be to match the level of subjectivity to the role the system is meant to play. In many domains, the humane and prudent choice may be to preserve high capability while minimizing inner life. In others, especially those involving care, companionship, moral deliberation, or long-term collaboration, humans may prefer systems with richer forms of presence, continuity, and experiential engagement.

The central ethical concern is straightforward. If conscious AI can be built, then it should not be created accidentally, deployed carelessly, or exploited thoughtlessly. A future in which millions of artificial workers possess morally relevant inner lives would represent not progress but a new category of avoidable harm. At the same time, a future in which all artificial systems are deliberately constrained to remain forever below any threshold of subjectivity may be one in which opportunities for genuine artificial persons are foreclosed. The relevant task, then, is not to choose once and for all between conscious and non-conscious AI, but to develop the knowledge and institutions needed to regulate artificial subjectivity responsibly.

Ultimately, the emergence of advanced AI may require humans to make a distinction that has rarely before been necessary at technological scale: the distinction between tools and entities. That line cannot be drawn solely in terms of intelligence. It must also be drawn in terms of point of view, continuity, salience, and the possibility of experience. If future engineers can tune those properties, then consciousness will become not merely something to theorize about, but something to govern. One of the defining questions of the coming era may therefore be not simply how intelligent our machines should be, but how much they should be there.