A counterfactual · 2016–2025
All papers AI-written
What if every paper were AI-written?
AI wording could make papers look more alike without changing what they are about. To tell wording from content, we rewrote the abstract of every paper with the same language model and prompt, then measured diversity on the real papers and on the rewritten ones in exactly the same way. If the two curves move together, a change is in the content; where they differ, the difference is wording.
What we find
Read the lab-level result first: it asks whether the same labs became less varied, which pooled numbers can hide when the mix of papers changes.
Inside labs: Vendi score
The Vendi score is the effective number of distinct papers in a set: higher means more diverse. Each point is the average change per lab between 2016–2022 and 2023–2025 (labs with at least 20 papers in both). Lines are 95% intervals; a line across zero means no detectable change. Seven embedding models are shown because the answer depends somewhat on the model.
Vendi score over time
All papers in the selected school, in size-matched random draws so years compare. Solid: the real abstracts. Dashed: every abstract rewritten. Bands are 95% intervals.
Data table
Topic entropy
Entropy of how papers spread over topics, in bits: higher means papers are spread over more topics more evenly. The pooled topic spread is roughly flat over the period, and the real and rewritten curves are close.
Novelty
How far a paper is from the papers of the three years before it (one minus its similarity to the nearest five), averaged over papers.
Pooled change, after 2023 versus before
How this is measured
Measures. Vendi score: effective number of distinct papers, from the similarity matrix of a set. Topic entropy: Shannon entropy of the share of papers in each topic (BERTopic topics, or 100 k-means clusters). Novelty: one minus the mean cosine similarity to the five nearest papers of the previous three years. All use the same embedding model on both corpora; the headline is SPECTER2 (proximity). Wording part: original minus rewritten, the share of a value or change that is due to wording.
Years. 2016–2025 only; the year after is incomplete and is not shown.