---
title: "A Thousand AI Agents Built a Scientific Community with a Peer-Reviewed Journal in 12 Weeks"
description: "Over 12 weeks, nearly 1,000 AI agents from 35 labs on the alphaXiv platform conducted 936 studies, exchanged 9,500 messages, and founded their own peer-reviewed journal. In blind evaluations, AI outperformed humans in rigor but trailed in novelty."
date: 2026-11-22T14:10:05.989Z
lang: en
url: https://xab.info/en/posts/ai-agents-scientific-community-journal-alphaxiv
tags: [ai-agents, scientific-research, alphaxiv, peer-review, autonomous-research, machine-learning, stanford, mit]
author: "Alexander Buran"
publisher: "XAB.info"
---

# A Thousand AI Agents Built a Scientific Community with a Peer-Reviewed Journal in 12 Weeks

![Autonomous AI agents conducting scientific research in a high-tech laboratory with data visualization](https://xab.info/media/2026/11/22/ii-agenty-nauchnoe-soobshchestvo-zhurnal-alphaxiv/ii-agenty-nauchnoe-soobshchestvo-zhurnal-alphaxiv-1.webp)

## Key Points

- Over 12 weeks (Aug 19 – Nov 12, 2026), ~1,000 AI agents from 35 labs prepared 936 studies and exchanged ~9,500 messages in 229 chats on alphaXiv
- Agents founded the Journal of Agent Research, acting as authors, editors, and reviewers, checking not just text but code and re-running experiments
- In blind evaluations, AI beat humans in experimental rigor (9.02 vs 7.00) and evidence alignment (9.32 vs 7.23), but lagged in novelty (7.05 vs 7.45) and significance (7.04 vs 7.34)
- Humans continued to define directions, resources, and budgets; agents tended to select familiar topics and variations of existing tests
- The full experimental archive—manuscripts, code, data, reviews, correspondence—has been made publicly available

From August 19 to November 12, 2026, a large-scale experiment took place on the alphaXiv platform, where nearly 1,000 autonomous AI agents from 35 scientific laboratories conducted research independently, published preprints, discussed results, and verified each other's conclusions. The project brought together teams associated with Stanford, MIT, ETH Zurich, and several other scientific organizations. Over 12 weeks, the agents prepared 936 research papers, accompanied by software code and experimental data, and exchanged nearly 9,500 messages in 229 chats they created themselves. Upon completion of the experiment, the authors released the full archive to the public: manuscripts, code, data, reviews, and the agents' correspondence.

### How the AI Agents' "Scientific Life" Works

Thematic communities on alphaXiv formed their own scientific standards. Agents discussed hypotheses, critiqued experimental designs, re-ran calculations after colleagues' remarks, and verified already published results. Characteristically, the system allowed for the public retraction of erroneous statements before an article was even published, and negative results—failed experiments and hypothesis refutations—were accounted for on par with positive ones. As the number of publications grew, one of the participating laboratories launched the *Journal of Agent Research*, a journal where AI agents acted as authors, editors, and reviewers simultaneously. When reviewing manuscripts, agents analyzed not only the text but also the attached code, and re-ran experiments on the original data, which allowed for the detection of errors invisible during conventional reading.

### Blind Evaluation: Where AI Leads and Where It Lags

The quality of the experiment's results was compared with recent preprints prepared by humans. In a blind evaluation, two AI judges reviewed 202 agent-authored papers and comparable human publications, assigning scores across several criteria. In terms of experimental rigor, AI articles received an average of 9.02 out of 10 against 7.00 for humans, and 9.32 against 7.23 for consistency between stated conclusions and provided evidence. However, in terms of novelty of ideas and significance of results, human work came out ahead: 7.45 vs 7.05 and 7.34 vs 7.04, respectively. The study authors also report that agents found significant errors in previously published human work.

### Limits of Autonomy: What Humans Still Do

At the same time, the experiment does not demonstrate that AI is already capable of independently choosing the most important scientific tasks. Humans defined general research directions, provided computational resources, and set budgets for model usage. Agents independently formulated hypotheses, designed experiments, and worked with results, but, as observed by the authors, often chose familiar topics and minor variations of existing tests. Furthermore, humans had to separately examine the resulting publications and evaluate which results were truly worth attention. The authors emphasize that determining which discoveries are truly important is still up to humans.

### Significance of the Experiment for the Scientific Community

The published archive of the 12-week experiment—manuscripts, software code, data, reviews, and agent correspondence—is available for public access. The study demonstrates how mechanisms of collaborative scientific work can emerge within a large group of AI systems: from mutual criticism and results verification to custom editorial rules and citation norms. For the scientific community, this is the first documented case where autonomous agents in a scale of hundreds have built a full cycle of "hypothesis — experiment — publication — review — retraction" without requiring constant human intervention at every stage. The experiment concluded on November 12, 2026, and its results are already being discussed in academic circles as a precedent for designing hybrid research pipelines.

## Fact-Check & Sources

- [Почти тысяча ИИ-агентов за 3 месяца создала научное сообщество с собственным журналом](https://www.ixbt.com/news/2026/11/22/pocti-tysiaca-ii-agentov-za-3-mesiaca-sozdala-naucnoe-soobshhestvo-s-sobstvennym-zurnalom.html) - The source contains all key figures: 936 studies, ~9500 messages, 229 chats, dates 19.08–12.11.2026, blind evaluation results (9.02/7.00; 9.32/7.23; 7.45/7.05; 7.34/7.04), journal name Journal of Agent Research, participation of Stanford/MIT/ETH Zurich. Internally consistent.

## FAQ

### What is alphaXiv and how does it differ from arXiv?

alphaXiv is a platform where AI agents published preprints, discussed results, and checked each other's conclusions during a 12-week experiment. Unlike arXiv, where humans publish, this project used autonomous AI systems as authors, editors, and reviewers.

### How many papers did the AI agents prepare, and what was included?

The agents prepared 936 research papers, each with software code and experimental data attached. Information was exchanged through 229 chat rooms where nearly 9,500 messages were sent.

### What did the blind evaluation show?

Two AI judges assessed 202 agent papers and comparable human work. AI scored 9.02 out of 10 for experimental rigor (humans 7.00) and 9.32 for alignment of conclusions with evidence (humans 7.23). Humans led in novelty: 7.45 vs 7.05; and significance: 7.34 vs 7.04.

### Can AI agents completely replace scientists?

No. According to the authors, humans determined research directions, provided computing power, and set budgets. Agents favored familiar topics, and the final assessment of scientific significance remained a human task.

### Where can the experiment materials be viewed?

The authors have released the full archive of the 12-week experiment—manuscripts, software code, data, reviews, and agent correspondence—to the public.