Abstract

Scientific knowledge is distributed across disciplines, publications, technical vocabularies, datasets, and communities whose members attend to only a small portion of the total conceptual landscape. Many potentially valuable ideas may therefore remain undiscovered even when most of their informational prerequisites already exist. The missing step may involve connecting separated concepts, identifying a shared mechanism, naming a recurrent but unrecognized pattern, reversing a familiar causal interpretation, or asking a question that no research community has prioritized. Large language models are unusually well positioned to search this latent scientific frontier because they encode broad statistical relationships across domains and can rapidly perform conceptual synthesis, literature retrieval, criticism, terminology formation, and theoretical elaboration. Ordinary prompting, however, tends to sample familiar, high-probability associations and usually depends on a human to supply the initial research question.

This article proposes recursive conceptual prospecting, an open-ended AI architecture that generates random and strategically semirandom combinations of scientific concepts, evaluates them for explanatory and empirical potential, investigates promising combinations in depth, and recursively explores their conceptual neighborhoods. The initial perturbation process is termed stochastic conceptual seeding. Productive seeds undergo rabbit-hole deepening, including domain acquisition, prior-art search, mechanism construction, adversarial criticism, prediction generation, and empirical testing. Discoveries and failures are consolidated into a synthetic frontier corpus, a structured repository containing hypotheses, evidence maps, terminology, search trajectories, rejected branches, confidence states, and unresolved questions.

The proposed system extends literature-based discovery, AI-supported ideation, multi-agent scientific systems, and open-ended hypothesis search. Its distinctive objective is global and cumulative exploration of underexamined conceptual space rather than completion of a single human-specified project. The synthetic frontier corpus would provide high-value process data for training future research systems and could serve as a developmental precursor to the Final Library, a cumulative machine-generated repository of scientific and conceptual knowledge. Strong provenance, independent evidence retrieval, negative-result preservation, adversarial evaluation, and explicit epistemic labeling would be necessary to prevent fluent speculation from being mistaken for discovery. Recursive conceptual prospecting could harvest substantial low-hanging conceptual fruit while gradually constructing the memory and agenda-forming infrastructure required for increasingly autonomous science.

Keywords: artificial intelligence, scientific discovery, large language models, hypothesis generation, literature-based discovery, computational creativity, synthetic data, autonomous science, concept combination, Final Library

1. Introduction

Scientific discovery is often associated with new observations, instruments, experiments, and mathematical techniques. Many important advances, however, depend on recognizing a relationship among facts that were already publicly available. Don Swanson’s classic investigation of fish oil and Raynaud syndrome demonstrated this possibility through what he called undiscovered public knowledge. Separate literatures described the effects of fish oil and the pathophysiology of Raynaud syndrome, but the potentially therapeutic relationship between them had not been explicitly developed. The discovery problem arose partly from the separation of scholarly communities rather than the absence of relevant information. 

The amount of available scientific knowledge has increased far beyond the attention span of any individual researcher. Specialization allows scientists to develop deep expertise, but it also partitions the intellectual landscape. Terminological differences, publication practices, disciplinary incentives, and limited reading time prevent many potentially meaningful connections from being considered. Even closely related concepts may remain separated if they are discussed by different fields, described at different levels of analysis, or embedded in unrelated research traditions.

Research on scientific innovation suggests that unusual combinations of existing knowledge can be disproportionately valuable. Uzzi and colleagues found that high-impact papers commonly combined a strong conventional foundation with a smaller contribution from atypical combinations. Shi and Evans later showed across large corpora of papers and patents that unexpected combinations of research content and disciplinary context predicted an increased probability of outsized impact. Research on higher-order novelty similarly treats the first occurrence of concept combinations as a distinct form of innovation that cannot be reduced to the appearance of new individual elements. 

The possible combinations grow much faster than the number of concepts. A vocabulary containing 100,000 scientific concepts permits approximately 5 billion unordered pairs and more than 166 trillion unordered triples. Most combinations are meaningless, redundant, or scientifically irrelevant. Yet humanity has not investigated more than a minute fraction of them, especially when the search includes relations among mechanisms, populations, time scales, ecological conditions, technologies, institutions, and levels of organization.

Large language models provide a possible means of searching this space. They can relate concepts across disciplines, translate among technical vocabularies, produce causal hypotheses, recognize analogies, propose terminology, retrieve literature, and transform a compressed initial idea into a larger theoretical structure. Current systems have already generated research proposals judged novel by experts, iteratively refined ideas using reviewing agents, synthesized evidence from tens of millions of papers, and conducted parts or all of computational research workflows. 

Their capacity for conceptual search remains underused. Most AI research systems begin with a paper, dataset, research goal, or problem supplied by a person. The system may explore within that region, but a human has already selected the region. The larger possibility considered here is an AI system that continuously initiates its own exploratory searches across the conceptual landscape.

The proposed process is called recursive conceptual prospecting. It treats scientific ideation as an allocation problem across a vast terrain. The system repeatedly samples combinations of concepts, asks whether they imply an unexamined relationship, rejects most combinations rapidly, and assigns larger research budgets to the minority that appear productive. A promising relationship becomes the entrance to a rabbit hole. The system acquires local expertise, searches for intellectual precursors, develops mechanisms, creates predictions, invites adversarial criticism, and investigates related combinations in the surrounding conceptual neighborhood.

The outputs would be preserved in a synthetic frontier corpus. This corpus would contain far more than polished articles. It would record what was searched, which connections were considered, what evidence was found, which branches failed, why they failed, how confidence changed, and what observations could resolve remaining uncertainty. These records could become training data for systems that learn how to conduct scientific search rather than merely how to imitate scientific writing.

Reser previously proposed the Final Library as a future repository containing machine-generated insights organized into recursively expanded, continuously updated conceptual structures. In that account, AI systems search large combinatorial spaces, evaluate candidate insights, and preserve the resulting synthetic knowledge in a repository too extensive for direct human navigation. Recursive conceptual prospecting can be understood as an initial generation engine for such a library, while the synthetic frontier corpus provides its first structured developmental layer. 

2. The Latent Scientific Frontier

The latent scientific frontier consists of potentially valuable ideas that are reachable from presently available knowledge but have not yet received adequate conceptual development, empirical evaluation, or scholarly attention. An idea belongs to this frontier when its main prerequisites are already accessible, while some missing act of synthesis, questioning, naming, modeling, or inference prevents it from becoming established knowledge.

This definition distinguishes conceptual accessibility from empirical truth. An AI may easily formulate a hypothesis that remains difficult to test. Some low-hanging conceptual fruit may require years of experimental work before its validity can be determined. The fruit is low hanging in the space of ideas because the hypothesis can be reached with existing knowledge, even when verification remains costly.

Several forms of output may occupy the latent frontier. A relational discovery identifies a previously neglected association or causal connection. A synthetic discovery unifies observations from separate literatures under one framework. A mechanistic discovery proposes an intervening process that explains an established correlation. A categorical discovery identifies a recurrent phenomenon that lacks a stable name or classification. A predictive discovery derives an unrecognized consequence from accepted premises. A methodological discovery identifies a measurement, comparison, or experimental manipulation capable of resolving an existing question.

Terminology can itself contribute to scientific progress. A term compresses a recurring pattern into a manipulable conceptual unit. Once a phenomenon is named, researchers can search for it, compare examples, dispute its boundaries, and connect it to other concepts. Coining terminology does not establish that a phenomenon is real, novel, or important, but appropriate naming can stabilize a previously diffuse insight and make cumulative analysis possible.

The frontier is generated partly by the architecture of human attention. Academic researchers operate within career incentives, grant systems, disciplinary vocabularies, methodological traditions, and local networks of collaborators. These structures direct attention toward tractable and institutionally legible questions. Large-scale analyses indicate that papers and patents have become less disruptive over time and draw on increasingly narrow portions of prior knowledge. More recent work examining 41.3 million natural-science papers found that AI-assisted research was associated with large individual productivity and citation advantages while collective scientific attention contracted toward data-rich regions. 

These patterns do not show that humanity is running out of ideas. They suggest that scientific search may be concentrated within familiar and well-supported areas. A system designed to seek underexamined combinations could counteract this tendency by allocating some resources according to conceptual neglect rather than current publication volume.

The latent frontier is also shaped by missing negative knowledge. Published literature contains many positive associations and successful methods, while failed experiments, null results, abandoned interpretations, and tacit laboratory judgments are less likely to be recorded. A 2026 expert audit involving 6,749 scientists found that contemporary LLMs rarely generated null hypotheses and that automated evaluators agreed only weakly with expert judgments. The authors argued that models inherit a literature shaped by publication bias and therefore lack much of the negative knowledge retained in scientific practice. 

A productive frontier corpus must consequently include known failures as well as promising ideas. A record showing that a conceptual combination was investigated and rejected can prevent repeated rediscovery of the same dead end. It can also preserve the conditions of failure, allowing the hypothesis to be reopened if new data, tools, or theoretical developments later alter those conditions.

3. Intellectual Precursors

3.1 Blind variation and selective retention

The basic logic of prospecting has a long history in theories of creativity. Campbell described creative thought and knowledge growth as processes involving variation followed by selective retention. Candidate ideas need not be produced through uniformly informed search. A system can generate partially blind variations and use subsequent evaluation to retain the small subset that exhibits value. 

Recursive conceptual prospecting instantiates this logic computationally. Stochastic seeding creates variation. Epistemic triage, retrieval, criticism, and experimentation provide selection. The frontier corpus supplies retention. Recursion occurs when retained products alter the system’s future concept inventory, evaluations, and sampling strategy.

The proposed variations are only partly blind. The system knows something about the concepts being combined, their semantic distance, their disciplinary location, and their existing connections. Randomness acts as a perturbation that prevents the model from repeatedly selecting the most probable associations. Evaluation remains highly informed.

3.2 Combinatorial and exploratory creativity

Scientific creativity includes combination of known concepts, exploration within existing conceptual systems, and transformation of the systems that define what counts as a possible idea. Recursive conceptual prospecting initially emphasizes combinatorial and exploratory creativity. It combines conceptual elements and follows the implications of productive combinations through structured search.

Transformational creativity remains possible when the system begins modifying its own ontologies, causal primitives, relation operators, or criteria of scientific relevance. A search may reveal that the existing concept graph represents a problem incorrectly. The system could then propose a new level of analysis, replace a category, divide one construct into several, or unify previously separate entities. Recent computational work has begun formalizing transformational scientific creativity as modification of the enabling assumptions that structure a conceptual space. 

3.3 Literature-based discovery

Literature-based discovery searches for connections among findings that are separately documented but not jointly developed. Swanson’s work established the classic A-B-C pattern: one literature relates A to B, another relates B to C, and the possible A-C relationship remains unexamined. Subsequent systems have expanded this approach through text mining, knowledge graphs, and biomedical relation extraction. 

Recursive conceptual prospecting extends the literature-based approach in several directions. It does not require a human to specify A or C. It searches beyond chains of explicit biomedical relations to include analogies, inversions, classifications, developmental sequences, economic processes, evolutionary functions, and cross-level mechanisms. It also treats the first productive connection as a point from which an entire discovery neighborhood can be explored.

3.4 Contemporary AI ideation systems

ResearchAgent begins with scientific literature and iteratively develops ideas using retrieval and reviewing agents. Deep Ideation searches a scientific concept network through an explore-expand-evolve workflow and maintains an Idea Stack to track candidate development. Both demonstrate that iterative retrieval and criticism can improve AI-generated research proposals. 

Co-Scientist uses multiple agents to generate, critique, rank, and refine biomedical hypotheses. Its hypotheses contributed to experimentally validated drug-repurposing candidates, target discovery, and an account of a bacterial gene-transfer mechanism. The AI Scientist integrates ideation, literature search, coding, experimentation, analysis, manuscript writing, and automated reviewing in machine-learning research. 

AutoDiscovery moves closer to autonomous initiation. It searches nested hypotheses over datasets using Monte Carlo tree search and Bayesian surprise rather than relying entirely on a human-specified question. Across 21 datasets, it produced more hypotheses judged surprising than comparison systems, and domain experts also considered many of its discoveries surprising. 

These systems establish many of the necessary components. The proposed contribution of recursive conceptual prospecting is their integration into a persistent, cross-disciplinary search process that samples new conceptual regions without waiting for a specific paper, dataset, or human research goal. To our knowledge, current published systems do not yet combine semantically stratified concept collision, recursive local domain acquisition, systematic neighborhood exploration, explicit preservation of dry holes, and consolidation into a cumulative frontier corpus intended to guide the next generation of searches. This novelty claim concerns the architecture as a whole rather than any individual component.

4. Why Stochastic Seeding Is Necessary

Language models do not sample uniformly from everything represented in their parameters. Their outputs are biased toward common continuations, familiar associations, and concepts occupying high-probability regions of the learned distribution. Repeated prompts can therefore produce variations of similar ideas while leaving the long tail underexplored.

Agrawal and Goyal systematically tested the addition of unrelated random concepts to prompts and found increased response diversity and entropy across multiple language models. Their experiments used list-generation tasks rather than scientific discovery, and the observed gains were sometimes modest. The results nevertheless support the underlying mechanism: an unrelated concept can perturb generation away from dominant response modes. 

Other studies reinforce the diversity problem. A blinded comparison involving more than 100 NLP researchers found that LLM-generated research ideas were judged more novel on average than human-generated ideas, while being slightly weaker in feasibility. The same study identified poor model self-evaluation and low diversity across repeated generations as important limitations. 

Deng and colleagues found that independent human samples often produce more diverse ideas than independent LLM samples because humans occupy different regions of knowledge space, while models aggregate information into a more unified distribution. They reported that structured reasoning and varied persona cues could increase idea diversity. A large 2026 scientist-in-the-loop evaluation similarly found that reasoning models explored a wider hypothesis space than non-reasoning models, although important omissions and weaknesses remained. 

Stochastic conceptual seeding is intended to create artificial knowledge partitioning. Instead of repeatedly asking one model to produce its most plausible scientific ideas, the system anchors separate searches in different combinations of concepts, disciplinary perspectives, scales, and relation types. Each seed creates a local research context that can diverge from the model’s default trajectory.

Uniform randomness would be inefficient. Randomly drawing arbitrary dictionary words would produce enormous quantities of decorative analogy, wordplay, and incoherent speculation. The system should use semantically stratified randomness, with separate sampling distributions for near, intermediate, and distant concepts.

Near combinations connect concepts already located in related literatures. They should yield a relatively high proportion of coherent ideas, although many will be incremental or previously developed. Intermediate combinations join concepts separated enough to escape conventional treatment while retaining plausible shared mechanisms. These may provide the highest average epistemic yield. Distant combinations will fail frequently but may occasionally produce unusually original theories or reveal that two fields instantiate the same higher-order structure.

The system should also sample across ontological categories. It might combine a disease phenotype with an animal adaptation, a cognitive mechanism with an economic institution, a developmental process with an AI failure mode, a technological constraint with a labor-market structure, or an ecological pressure with an age-related physiological change. Cross-category combinations reduce the probability that the output merely reproduces an established within-field question.

5. Recursive Conceptual Prospecting

Recursive conceptual prospecting is an open-ended process in which an AI system generates semirandom conceptual combinations, evaluates them for underdeveloped scientific potential, investigates promising combinations through increasingly deep research, searches neighboring combinations, and incorporates the resulting knowledge into the structure that guides future exploration.

The process is recursive in three senses. First, a promising idea causes the system to prompt itself with increasingly specific questions. Second, each discovered mechanism or distinction generates new concepts that can be recombined. Third, the accumulated frontier corpus changes the probability distribution from which future searches are selected.

5.1 Concept inventory construction

The system begins with a structured inventory containing scientific entities, processes, mechanisms, methods, measurements, populations, environments, pathologies, technologies, theories, and institutions. Concepts should be linked to definitions, synonyms, disciplinary contexts, dates, source literatures, and levels of abstraction.

A concept graph derived only from co-occurrence would be insufficient. Two terms can co-occur for many reasons, and important relationships may appear without repeated lexical proximity. The graph should encode causal, functional, temporal, taxonomic, analogical, inhibitory, enabling, and evidential relations wherever these can be recovered.

Retrieval systems such as OpenScholar show the scale at which scientific evidence can now be indexed and synthesized. OpenScholar retrieves from 45 million open-access papers and produces citation-backed scientific responses, illustrating the feasibility of connecting generative models to a large, externally verifiable literature store rather than relying on model memory alone. 

5.2 Stochastic conceptual seeding

Stochastic conceptual seeding is the deliberate insertion of random or semirandom concepts into an AI’s context to initiate exploration outside its default associations. A seed may contain two concepts, but higher-order combinations will often be useful.

Each seed should be annotated with semantic distance, disciplinary distance, ontological type, publication density, and prior co-occurrence. Sampling can draw from several policies:

\pi(s)
=
\alpha \pi_{\text{near}}(s)
+
\beta \pi_{\text{intermediate}}(s)
+
\gamma \pi_{\text{distant}}(s)
+
\delta \pi_{\text{underexplored}}(s)
+
\epsilon \pi_{\text{anomaly}}(s),

where s is a seed combination and the mixture weights determine how much compute is assigned to each search mode.

The underexplored policy samples concept pairs or triples that appear individually in mature literatures but rarely appear together. The anomaly policy begins with unexplained findings, conflicting studies, unexpected absences, or phenomena poorly predicted by prevailing theories.

5.3 Relation operators

Concepts do not become hypotheses merely by appearing in the same prompt. The system needs operators that ask what form of relationship could connect them. Relevant operators include causation, common mechanism, functional equivalence, opposition, substitution, developmental sequence, evolutionary adaptation, constraint, feedback, scaling, analogy, category membership, measurement, and null association.

A single seed should be examined through several operators. If the concepts are “sleep fragmentation” and “immune resource allocation,” the system might ask whether one causes the other, whether both arise from a common process, whether fragmentation serves an adaptive function under certain conditions, whether immune state moderates the relationship, or whether the expected association is absent in a revealing population.

The output of this stage is a set of concise candidate propositions. Each proposition should specify a relation, mechanism where possible, applicable conditions, and an observation that would count against it.

5.4 Epistemic triage

Most candidates should be rejected quickly. Epistemic triage determines whether a conceptual collision merits further research.

The initial filters should examine coherence, nontriviality, mechanistic plausibility, potential importance, testability, and obvious duplication. Candidates fail if they consist only of metaphor, depend on a category error, merely restate definitions, conflict with well-established constraints without offering an explanation, or remain too vague to produce discriminating predictions.

The system should assign explicit reasons for rejection. A failed branch can be classified as incoherent, already established, superficially novel, contradicted, currently untestable, low-value, safety-restricted, or dependent on unavailable evidence. These classifications become part of the corpus.

5.5 Prior-art search

Novel wording provides little evidence of conceptual novelty. The system must search for the underlying proposition using synonyms, paraphrases, neighboring mechanisms, older terminology, citations, patents, dissertations, conference papers, books, and non-English sources.

Prior art should be classified along a continuum:

1. Established: the concept and mechanism are already well developed.
2. Named but underdeveloped: a term exists, but the theory or evidence remains limited.
3. Partially anticipated: one or more components have been proposed without the complete synthesis.
4. New integration: the components are established, while their organization into the present framework appears new.
5. Apparently novel hypothesis: no substantive precedent is found after broad search.
6. Novelty unresolved: available retrieval is insufficient for a dependable judgment.

This stage prevents terminological novelty inflation, in which an old concept receives a new name and is mistakenly treated as a discovery. It also allows the system to recover neglected predecessors and give them proper intellectual credit.

5.6 Rabbit-hole deepening

A surviving candidate enters rabbit-hole deepening, the transition from a speculative connection to concentrated domain acquisition and theory development. The system reads relevant literatures, learns technical vocabulary, constructs a causal model, searches for comparative evidence, identifies boundary conditions, and determines which findings should exist if the proposal is correct.

Deepening should produce a structured research object containing a thesis, conceptual definitions, proposed mechanism, evidence for and against, alternative explanations, predicted observations, discriminating tests, unresolved questions, and a calibrated confidence state.

The process can include multiple specialized agents. A synthesizer develops the theory, a retriever gathers evidence, a historian searches prior art, a methodologist designs tests, a statistician examines identifiability, and a critic attempts to falsify the proposal. ResearchAgent, Co-Scientist, and related systems provide precedents for using separated generative and evaluative roles. 

5.7 Discovery-neighborhood expansion

A successful rabbit hole should not terminate with one article. It should initiate discovery-neighborhood expansion, a systematic search of nearby conceptual combinations.

The system can vary the species, population, disease, institution, historical period, technological substrate, level of organization, causal direction, and time scale. It can ask whether the proposed mechanism has analogues in neighboring domains, whether it predicts a broader category, or whether the relationship reverses under different conditions.

This stage converts an isolated insight into a local map. Productive neighborhoods may develop their own terminology, taxonomies, competing theories, and research programs. Unproductive neighboring searches help define the boundary of the original proposal.

5.8 An illustrative conceptual trajectory

Consider a seed combining cognitive automation, humanoid-robot scarcity, and gig labor. A first inference is that artificial intelligence may become capable of directing physical work before enough robotic bodies exist to perform it. Human beings may then temporarily fill work positions designed for robots.

Rabbit-hole deepening yields the embodiment lag, the period between abundant artificial cognition and abundant artificial bodies. Further analysis produces anticipatory robotification, reverse robotic substitution, robot slots, AI-directed embodiment labor, and the embodiment gig economy. Neighborhood expansion raises questions about augmented-reality instruction, machine-derived work standards, human demonstration data, robot-training compensation, and the eventual withdrawal of human-compatible workplace modes.

The initial combination is compact, while its conceptual neighborhood contains an entire theoretical program. In this case, a human supplied the seed and AI assisted with decompression, terminology, research, and implication search. Recursive conceptual prospecting attempts to automate the production of candidate seeds while preserving the deepening process.

6. A Formal Search Model

Let the current conceptual environment be represented by a dynamic graph:

G_t=(V_t,E_t),

where V_t contains concepts and E_t contains known or proposed relations at time t. A seed s is a subset of V_t, usually containing two to five concepts.

A generator applies relation operators R to produce a set of candidate hypotheses:

H(s)=\{h_1,h_2,\ldots,h_n\}.

Each hypothesis receives preliminary estimates for conceptual novelty N, mechanistic plausibility P, scientific importance I, empirical testability T, evidence accessibility E, redundancy with prior art D, research cost C, and safety risk S. A provisional prospecting score can be written as:

Q(h)
=
\frac{
N(h)^{w_N}
P(h)^{w_P}
I(h)^{w_I}
T(h)^{w_T}
E(h)^{w_E}
}{
[1+D(h)]^{w_D}
[1+C(h)]^{w_C}
}
-
w_S S(h).

The weights depend on the research agenda. A system devoted to fundamental theory may assign more weight to novelty and explanatory reach. A biomedical system may emphasize evidential accessibility, clinical importance, and safety.

The estimates are uncertain and should be represented as distributions. The value of additional investigation depends on how much it could change the system’s beliefs and whether that change would influence future decisions. A more principled allocation criterion is expected epistemic value:

EEV(h,a)
=
\frac{
\mathbb{E}[\Delta U(h)\mid a]
}{
\operatorname{Cost}(a)
},

where a is a research action and \Delta U(h) is the expected improvement in the scientific utility assigned to the hypothesis after performing that action.

The system can allocate compute through a bandit algorithm, tree search, or progressive widening. Most branches receive little investment. Branches producing evidence, surprise, explanatory compression, or valuable neighboring questions receive increasing budgets. AutoDiscovery’s use of Bayesian surprise and Monte Carlo tree search provides one existing demonstration of how open-ended hypothesis search can be guided by epistemic change rather than a fixed external target. 

The graph changes whenever the system adds a term, rejects a proposed relation, validates a mechanism, or discovers a new distinction. Consequently:

G_{t+1}
=
G_t
+
\Delta G_{\text{discovery}}
+
\Delta G_{\text{negative}}
+
\Delta G_{\text{reorganization}}.

Future search occurs within a conceptual environment altered by earlier search. This cumulative modification distinguishes recursive conceptual prospecting from repeatedly issuing independent random prompts.

7. Preventing the System from Going Off the Rails

The central danger is that language models can create plausible explanations for arbitrary combinations. Fluency allows weak analogies and invented mechanisms to resemble scientific reasoning. A high-throughput prospecting system without rigorous adjudication would produce a vast collection of persuasive falsehoods.

The architecture must therefore separate generation from warrant. A generated idea is evidence that a proposition can be coherently articulated. It provides no direct evidence that the proposition is true.

7.1 Independence of imagination and adjudication

The agent proposing a relationship should not be the sole evaluator of its value. Separate agents or model instances should conduct prior-art search, evidence retrieval, causal criticism, statistical review, and safety evaluation. Their contexts should be designed to reduce shared anchoring on the original explanation.

Where feasible, evaluators should be blinded to persuasive framing and receive the hypothesis in a standardized form. They should examine claims individually rather than rate the rhetorical quality of a complete article.

Current evidence makes this separation essential. Si and colleagues found failures of LLM self-evaluation even when AI ideas were rated as relatively novel by experts. Bao and colleagues found weak agreement between automated evaluators and scientists across novelty, feasibility, and probability, although reward models trained on expert ratings improved performance. 

7.2 Retrieval-grounded claim ledgers

Every factual claim should be entered into a claim ledger with its supporting sources, source type, publication date, and evidential role. The system should distinguish direct evidence, indirect evidence, analogy, theoretical inference, and unsupported speculation.

Retrieval must prioritize primary evidence. Reviews and secondary summaries can guide search, but they should not become the terminal support for central empirical claims when original studies are available. Contradictory evidence should be recorded alongside supportive evidence rather than silently excluded.

OpenScholar demonstrates that specialized retrieval and iterative self-feedback can greatly improve citation accuracy relative to an unaided general model. It also documents how severe citation hallucination can be in general-purpose generation, supporting the need for externally grounded evidence systems. 

7.3 Epistemic status labels

Every corpus entry should carry an explicit status. Suitable categories include:

• speculative conceptual possibility;
• mechanistically plausible hypothesis;
• literature-supported synthesis;
• retrospectively supported prediction;
• prospectively tested result;
• independently replicated finding;
• contradicted hypothesis;
• abandoned hypothesis;
• dormant hypothesis awaiting new tools or evidence.

These labels prevent epistemic laundering, a process in which an AI-generated conjecture is stored as polished prose, retrieved later as apparent evidence, and eventually treated as established knowledge because multiple synthetic texts repeat it.

Synthetic sources should never count as independent empirical corroboration. The system must trace claims back to observations, experiments, formal derivations, or identifiable human scholarship. Repetition within the corpus can indicate conceptual interest, but it cannot increase evidential confidence by itself.

7.4 Active search for disconfirmation

Critic agents should attempt to identify observations that would falsify the proposal, established findings that contradict it, alternative mechanisms producing the same predictions, and conditions under which the relationship disappears. They should also generate null hypotheses and subtraction-based explanations, areas in which current systems appear comparatively weak. 

A hypothesis that survives criticism should not automatically be accepted. Survival raises its priority for empirical testing. The strongest output of conceptual prospecting is often a well-formed question or discriminating experiment rather than a conclusion.

7.5 Preserving failed searches

Dry holes are scientifically useful. Each failed exploration should retain the seed, relation operators, literature searched, rejection reason, and conditions under which the judgment was made.

This creates a negative knowledge ledger. The ledger prevents wasteful repetition, supplies examples for training evaluators, and reveals systematic regions where the model repeatedly produces category errors or superficial analogies. It can also identify blind spots in the evidence base.

A branch rejected as untestable should remain distinct from one rejected as false. Advances in measurement can make previously inaccessible hypotheses tractable. The frontier corpus should support reopening rather than permanent deletion.

7.6 Safety gating

Global conceptual search will inevitably encounter domains in which generating or elaborating ideas can create material risk. Biological, chemical, cyber, surveillance, and weapons-related searches require risk-sensitive policies, restricted tools, access controls, and human oversight.

Safety evaluation should occur before deepening, not only before publication. A system may need to preserve a minimal record that a region was encountered and restricted without retaining operational details that would increase misuse risk.

8. The Synthetic Frontier Corpus

A synthetic frontier corpus is a structured collection of AI-generated hypotheses, conceptual frameworks, new terminology, evidence maps, experimental proposals, critical evaluations, and failed searches focused on the boundary between established knowledge and underexplored possibility.

The corpus differs from an ordinary synthetic-text dataset. Its primary unit is a research trajectory rather than a finished passage. It records how the system moved from a seed to a candidate idea, how evidence changed the idea, which objections altered it, why neighboring branches were explored, and what caused the investigation to stop.

A complete entry would include the seed combination, semantic distances, relation operators, initial hypotheses, triage decisions, search queries, sources retrieved, claim ledger, prior-art classification, model revisions, critical objections, proposed tests, confidence history, branch genealogy, negative findings, safety classification, and final status.

Recent work on IdeaTrail illustrates the growing recognition that scientific ideation is a long-horizon process. IdeaTrail contains 1,170 multi-turn trajectories recording literature search, tool use, evidence gathering, claim checking, synthesis, rejection, and proposal construction. Recursive conceptual prospecting would extend this process-oriented approach from reconstruction around selected research artifacts to open-ended exploration whose endpoints are not known in advance. 

8.1 Why trajectories are valuable training data

A polished article shows what a successful endpoint looks like. It does not show which ideas were considered and abandoned, which search query found the decisive paper, how a causal mechanism changed, or when the researcher recognized that an apparently new idea already existed.

Trajectory data can train future systems to perform several higher-order functions: choosing productive seeds, varying semantic distance, recognizing a promising but initially weak idea, terminating sterile branches, distinguishing renaming from discovery, designing discriminating tests, calibrating confidence, and preserving negative knowledge.

The strongest training examples may be contrasts between superficially similar trajectories. One branch may fail because the concepts share only metaphorical similarity. Another may succeed because the same concepts are connected by a measurable mechanism. Training on both allows the model to learn what separates conceptual novelty from scientific value.

8.2 Synthetic data without synthetic detachment

The corpus should remain anchored to human-generated observations, primary literature, and real experimental outcomes. Research on recursive training has shown that indiscriminately training generative models on outputs from previous models can produce model collapse, including progressive loss of low-probability information from the original distribution. 

The frontier corpus should therefore function as an annotated reasoning layer over reality-based sources. It should not replace the empirical and human record. Every synthetic inference should retain provenance linking it to the evidence from which it arose, and every training mixture should preserve substantial access to original human and observational data.

The corpus could counteract model collapse when it deliberately preserves long-tail concepts, failed branches, rare hypotheses, null findings, and diverse disciplinary perspectives. Its value would derive from increasing structured variance while retaining source grounding.

8.3 Versioning and conceptual genealogy

Scientific ideas change. A corpus entry should be versioned, with every modification linked to new evidence or criticism. Competing versions can coexist when the issue remains unresolved.

Concepts should also have genealogies. If several independent prospecting runs reach similar ideas, the system should merge or cross-reference them while preserving their distinct routes. Independent convergence can increase confidence that a conceptual region is naturally reachable or important, although it still provides no empirical confirmation.

New terminology should be linked to prior terms and conceptual ancestors. This allows future users to determine whether a coined term represents a new phenomenon, a broader category, a narrower subtype, or a restatement of existing work.

9. From the Frontier Corpus to the Final Library

The synthetic frontier corpus and the Final Library serve related but distinct functions. The corpus records active exploration near the boundary of knowledge. The Final Library would integrate validated knowledge, unresolved hypotheses, conceptual genealogies, evidence, failures, and synthetic theories into a more comprehensive machine-readable intellectual memory.

The developmental sequence can be represented as:

\text{stochastic conceptual seeding}
\rightarrow
\text{recursive conceptual prospecting}
\rightarrow
\text{synthetic frontier corpus}
\rightarrow
\text{Final Library}
\rightarrow
\text{library-directed discovery}.

During the earliest stage, randomness compensates for the absence of a sufficiently developed autonomous research agenda. The system forces itself to visit conceptual locations that its normal probability distribution neglects.

As the corpus grows, exploration can become increasingly informed. The library can identify sparsely connected regions, recurring unresolved questions, theories lacking mechanisms, claims supported by disconnected evidence, contradictory findings, concepts repeatedly renamed, and old hypotheses made newly testable by technological change.

The Final Library would then provide library-driven epistemic nucleation. Instead of beginning with arbitrary word combinations, the system could generate research seeds from the topology of accumulated knowledge. Randomness would remain useful as a source of variation, but it would operate alongside increasingly sophisticated maps of epistemic opportunity.

Reser’s original Final Library proposal envisioned a repository containing machine-generated conceptual trees that are continuously expanded, tested, refined, and cross-referenced. Recursive conceptual prospecting supplies a practical route toward that repository by defining how candidate insights are generated and how the exploration process can be retained. 

The library alone would not create autonomous science. It would also require agenda formation, significance estimation, research-resource allocation, experimental access, and governance. The corpus provides memory and search experience. A complete autonomous research institution must also determine which discoveries are valuable, safe, and worthy of scarce physical resources.

10. An Empirical Research Program

Recursive conceptual prospecting is testable. Its success should be measured prospectively rather than inferred from persuasive examples produced after the fact.

10.1 Random concepts versus controlled conceptual distance

Researchers could compare four ideation conditions: standard prompting, uniformly random word infusion, semantically stratified concept combinations, and concept combinations selected from underexplored regions of a scientific graph.

Domain experts blinded to condition would rate novelty, plausibility, importance, testability, and resemblance to prior work. The central prediction is that uniformly random seeding will increase diversity but produce many low-quality candidates, while intermediate-distance and underexplored-edge sampling will yield a better novelty-validity balance.

10.2 The semantic-distance yield curve

The theory predicts a relationship between conceptual distance and discovery yield. Near combinations should produce many coherent but familiar ideas. Extremely distant combinations should produce many incoherent ideas. Intermediate combinations should provide the highest average rate of useful novelty.

The curve need not be identical across disciplines. Mathematics may tolerate larger conceptual distances when formal evaluation is available. Clinical medicine may require closer grounding because biological plausibility and safety impose stricter constraints.

10.3 Value of recursive deepening

A study could compare single-pass ideation against rabbit-hole deepening with fixed compute budgets. The latter condition would include iterative literature acquisition, critique, prior-art search, prediction generation, and neighborhood expansion.

The prediction is that deepening will reduce the number of surviving ideas while increasing their explanatory specificity, evidential grounding, and feasibility. ResearchAgent, Deep Ideation, Co-Scientist, and The AI Scientist already provide evidence that iterative refinement and greater test-time computation can improve scientific outputs. 

10.4 Neighborhood productivity

Productive seeds should generate clusters of related insights. Researchers can measure the number and quality of viable neighboring hypotheses emerging from each initial discovery.

A strong conceptual seed should produce more than paraphrases. It should reveal new mechanisms, boundary conditions, comparisons, measurements, applications, or classifications. This neighborhood productivity may be a better early estimate of theoretical importance than the rhetorical quality of the first article.

10.5 Novel terminology evaluation

Coined terms can be tested for usefulness. Experts can evaluate whether a term identifies a coherent phenomenon, improves conceptual compression, distinguishes previously conflated cases, and supports productive search.

Subsequent use provides another measure. A valuable term should help users generate new questions, classify observations, or connect literatures. A term that merely replaces established language will add little.

10.6 Prospective prediction and rediscovery

The strongest validation would involve timestamped hypotheses generated using a literature cutoff. Predictions would be registered before new evidence becomes available. Later findings could determine whether prospecting anticipated discoveries, proposed useful null results, or identified relationships independently rediscovered by human researchers.

Such testing would help separate historical reconstruction from genuine prospective inference. It would also create a growing benchmark of ideas whose outcomes are initially unknown.

10.7 Training on the frontier corpus

Models trained on full exploration trajectories could be compared with models trained only on successful final proposals. Evaluation should measure diversity, novelty, evidence use, calibration, branch termination, and resistance to terminological novelty inflation.

The prediction is that trajectories containing both successful and failed searches will improve scientific judgment more than a corpus containing polished successes alone. Negative examples should be especially helpful for teaching models when to abandon an attractive explanation.

10.8 Expansion versus concentration of scientific attention

A system designed for prospecting should expand the range of topics explored relative to conventional AI-supported science. This can be assessed through concept diversity, disciplinary distance, coverage of low-publication regions, and the distribution of research resources across data-rich and data-poor areas.

This test is important because present AI adoption may increase individual productivity while concentrating collective attention. A successful prospecting architecture should deliberately resist that tendency. 

11. Scientific and Institutional Implications

11.1 The bottleneck may move from ideation to validation

If AI can generate thousands or millions of plausible hypotheses, scientific scarcity shifts. The limited resources become expert attention, laboratory time, access to participants, physical instrumentation, longitudinal data, and replication capacity.

The system must therefore estimate the value of information produced by each experiment. Cheap ideation without disciplined prioritization could overwhelm scientific institutions with proposals that no one can test.

11.2 Articles may cease to be the primary unit of synthetic science

A paper is designed for human communication and credit assignment. It is poorly suited to representing millions of branching, partially evaluated hypotheses. The synthetic frontier corpus would require a structured claim-and-evidence format beneath the article layer.

Human-readable papers would still be generated when a theory merits extended explanation. The underlying research object would remain a versioned network of claims, sources, predictions, objections, and descendants.

11.3 Scientific attribution becomes more granular

A prospecting system may involve a human who defined the architecture, an AI that generated a seed, another model that coined terminology, a retrieval system that found evidence, a laboratory that tested the prediction, and researchers who interpreted the result.

Attribution should record these roles separately. Conceptual origination, computational elaboration, empirical validation, system engineering, and scholarly accountability are distinct contributions. Provenance within the corpus could preserve this division more accurately than a conventional author list.

11.4 Researchers may become frontier curators

Human scientists could spend less time producing initial candidate hypotheses and more time judging significance, identifying overlooked constraints, contributing tacit knowledge, selecting experiments, and deciding which research directions serve human values.

Current evidence still supports an important human role. Large expert studies find that AI judgments and hypothesis distributions remain limited, especially in fields requiring contextual interpretation or pluralistic theory. Recursive conceptual prospecting should initially be treated as a large-scale candidate generator and research collaborator rather than an unquestioned arbiter of scientific truth. 

11.5 Low-hanging fruit may be harvested rapidly

Once a reliable prospecting system operates at scale, many conceptually accessible ideas could be found within a relatively short period. This would not end science. It would change the distribution of remaining problems.

After easy syntheses and naming opportunities are exhausted, research would increasingly depend on difficult experimentation, new instruments, interactions with the physical world, and transformation of existing conceptual spaces. The Final Library would allow future systems to know which combinations have already been considered and direct effort toward deeper frontiers.

12. Limitations

The existence of a large combinatorial space does not imply that it contains a proportionally large number of valuable truths. Most combinations will remain irrelevant. The scientific value of prospecting depends on the efficiency of selection, not the volume of generated text.

Language models also inherit biases and omissions from the literature. They may be especially weak where knowledge is tacit, embodied, unpublished, politically suppressed, or expressed outside dominant scientific languages. Retrieval expands access but cannot recover information that was never recorded.

Novelty is difficult to establish. Literature search can fail to retrieve obscure precedents, especially when terminology differs radically. Every novelty classification should remain provisional and should be revised when earlier work is found.

Scientific evaluation cannot be reduced to a single score. Plausibility, importance, novelty, feasibility, and safety may conflict. Reward models can concentrate research around the preferences of their training community and suppress unconventional ideas.

Some discoveries depend on observations that cannot be inferred from textual knowledge. An AI cannot discover an unrecorded species, unexpected material property, or novel biological response through conceptual recombination alone. It can propose where to look, while the world retains the final capacity to surprise.

The frontier corpus could amplify errors if provenance and epistemic status are neglected. Recursive reuse of synthetic claims creates the risk of self-confirming knowledge systems detached from empirical reality. Primary evidence, prospective testing, independent replication, and preserved human data remain essential.

Finally, an autonomous prospecting system requires governance. The choice of research agenda reflects values. A system optimized only for novelty, publication impact, commercial value, or capability growth may neglect public health, environmental protection, neglected populations, and long-term social welfare.

13. Conclusion

Humanity has produced an immense body of knowledge while exploring only a small portion of the relationships among its concepts. Disciplinary separation, limited attention, institutional incentives, and the combinatorial growth of possible connections leave a latent scientific frontier composed of questions that have not been asked, mechanisms that have not been articulated, and patterns that have not been named.

Large language models provide a new search instrument for this frontier. They contain broad statistical representations of scientific language and can rapidly synthesize, elaborate, retrieve, criticize, and formalize ideas. Their ordinary response distributions remain concentrated around familiar associations, and they usually depend on humans to initiate research directions.

Recursive conceptual prospecting would convert these models from passive respondents into continuous explorers. Stochastic conceptual seeding would move them into less frequently sampled regions. Epistemic triage would discard most combinations. Rabbit-hole deepening would turn the most promising seeds into serious research programs. Discovery-neighborhood expansion would map the regions surrounding successful connections.

The synthetic frontier corpus would preserve both discoveries and failures as structured research trajectories. It would provide training data for future systems that learn how to search, judge, revise, and terminate scientific investigations. With sufficient provenance and empirical grounding, it could become the precursor to a Final Library containing the cumulative intellectual products of artificial science.

Random combination would serve as the bootstrap rather than the endpoint. As exploration accumulated, the library itself would identify gaps, contradictions, neglected regions, and newly tractable questions. The search process would gradually move from stochastic conceptual collisions toward informed navigation of an increasingly explicit epistemic landscape.

The central proposal can therefore be stated concisely:

Artificial intelligence can be used to mine the latent scientific frontier by repeatedly combining concepts, selectively deepening promising connections, and preserving the complete history of exploration in a cumulative synthetic corpus.

Many exploratory holes will be empty. A minority will yield useful terms, hypotheses, mechanisms, predictions, or research programs. At machine scale, even a low success rate could substantially enlarge the range of scientific ideas available for human and artificial evaluation. Recursive conceptual prospecting would make the search for previously unasked questions into a systematic scientific enterprise.

References

Agrawal, P., & Goyal, P. (2026). Addressing LLM diversity by infusing random concepts. arXiv:2601.18053. 

Agarwal, D., Majumder, B. P., Adamson, R., et al. (2025, revised 2026). AutoDiscovery: Open-ended scientific discovery via Bayesian surprise. arXiv:2507.00310. 

Asai, A., et al. (2026). Synthesizing scientific literature with retrieval-augmented language models. Nature, 650, 857–863. 

Baek, J., Jauhar, S. K., Cucerzan, S., & Hwang, S. J. (2025). ResearchAgent: Iterative research idea generation over scientific literature with large language models. Proceedings of NAACL 2025. 

Bao, H., Wu, S., Liu, X., et al. (2026). Contemporary AI lacks the imagination to diverge or negate in science. arXiv:2606.08251. 

Campbell, D. T. (1960). Blind variation and selective retention in creative thought as in other knowledge processes. Psychological Review, 67, 380–400. 

Deng, Y., Brucks, M., & Toubia, O. (2026). Examining and addressing barriers to diversity in LLM-generated ideas. arXiv:2602.20408. 

Di Bona, G., Bellina, A., De Marzo, G., et al. (2025). The dynamics of higher-order novelties. Nature Communications, 16, 393. 

Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10, eadn5290. 

Gottweis, J., et al. (2026). Accelerating scientific discovery with Co-Scientist. Nature, 655, 487–496. 

Guo, H. (2026). IdeaTrail: Full-process agent trajectories for scientific ideation. arXiv:2607.10144. 

Hao, Q., Xu, F., Yong, L., et al. (2026). Artificial intelligence tools expand scientists’ impact but contract science’s focus. Nature. 

Lu, C., et al. (2026). Towards end-to-end automation of AI research. Nature. 

Park, M., Leahey, E., & Funk, R. J. (2023). Papers and patents are becoming less disruptive over time. Nature, 613, 138–144. 

Reser, J. E. (2025). The Final Library and the last years of human-original thought. Iterated Insights. 

Shi, F., & Evans, J. (2023). Surprising combinations of research contents and contexts are related to impact and emerge with scientific outsiders from distant disciplines. Nature Communications, 14, 1641. 

Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024). AI models collapse when trained on recursively generated data. Nature, 631, 755–759. 

Si, C., Yang, D., & Hashimoto, T. (2024). Can LLMs generate novel research ideas? A large-scale human study with 100+ NLP researchers. arXiv:2409.04109. 

Swanson, D. R. (1986). Fish oil, Raynaud’s syndrome, and undiscovered public knowledge. Perspectives in Biology and Medicine, 30, 7–18. 

Uzzi, B., Mukherjee, S., Stringer, M., & Jones, B. (2013). Atypical combinations and scientific impact. Science, 342, 468–472. 

Zhao, K., Lin, W., Zheng, Q., Xu, F., & Li, Y. (2025). Deep Ideation: Designing LLM agents to generate novel research ideas on scientific concept networks. arXiv:2511.02238.