--- title: "Language models and idea diversity: homogenisation within a session, across users, and between companies" kind: "Article" topic: "AI-directed development" date: "2026-10-10" url: "https://canopticon.com/articles/language-models-and-idea-diversity" author: "Chris Telford" license: "CC BY 4.0" --- You ask an AI for a fresh angle on a problem. It gives you something that sounds sharp. You ask another model to check, and it agrees. You keep refining together for an hour, and the idea gets smoother and more confident. The research behind this article says each of those steps moves you toward the average, not away from it. ## Different models give much the same answer A second model feels like a second opinion. The largest test of this, over 70 models and 26,000 open questions, found "intra-model repetition" and, "more so," "inter-model homogeneity, where different models produce strikingly similar outputs." So switching to another model does not reliably give you a second opinion. Part of the reason is training itself: The model has been trained to hand you the most typical answer. ## AI ideas look novel, then shrink when built The most careful test of AI research ideas ran in two stages. In the first, with more than 100 researchers, reviewers judged model-generated ideas "more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility." Then the ideas were built. In the follow-up, 43 experts each spent over 100 hours executing randomly assigned ideas. Afterwards the model ideas' scores fell further than the human ones "on all evaluation metrics (novelty, excitement, effectiveness, and overall; p < 0.05)," and "for many metrics there is a flip in rankings where human ideas score higher than LLM ideas". The research calls this the empirical version of "plausible-sounding but hollow synthesis." The models are also poor judges of their own work. Si et al. found LLMs were unreliable at judging their own ideas. So asking the model whether its idea is original is the check least likely to help. ## Long sessions pull toward the model's favourite themes The longer you iterate with one model, the narrower it gets. Asked for new combinations of ideas, one leading model reused the same few concepts almost every time: its ten most-used concepts appeared in 95.7% of outputs. Over a run of research reports, its topics drifted toward one subject it had not started on. Other models showed the same pull to a lesser degree. For a second leading model the figure was 76.7%, versus 34.3% for a sampler that deliberately avoided "cognitively available" combinations. The drift over a long run was measurable: Claude's reports drifted from 2 of 10 on interpretability to 8 of 10, even though the ideas had not started there. The research flags the source's limits: the paper is a recent preprint, and its model-specific numbers describe particular model versions on one research corpus. ## Surprising ideas come from outsiders, and look wrong at first If the model drifts to the centre, the useful question is where surprising ideas usually come from. One study found that surprising combinations of research content and context are associated with higher impact, and that they tend to come from scientists crossing in from distant disciplines. Another found that the papers a model finds most perplexing have bimodal fates: they are over-represented among the most celebrated work and also among the most dismissed. The research draws the uncomfortable lesson: Radical catches will often look wrong at first. The same idea can be turned into a tool. By modelling which people are likely to make which inferences, researchers improved AI prediction of future discoveries by up to 400%, and by reversing the model to "avoid the crowd," produced promising "alien" hypotheses that people were unlikely to pursue. The test that follows has two parts: score ideas on two separate axes, coherence (does it hang together?) and availability (is anyone already positioned to say it?) ## It narrows the crowd, not only the person The effect is worse in aggregate. Writers given AI story ideas wrote more creative stories — and more similar ones. The researchers describe "an increase in individual creativity at the risk of losing collective novelty." If everyone in a field prompts the same few models, the field converges, and each person feels more original while it happens. > **Objection: But the idea feels new** > > That feeling is the least reliable part. A preregistered study found that an "Aha!" moment triggered by solving an unrelated anagram made false statements feel truer. The rush of insight is the same whether the idea is good or not, so it cannot be the test. > > It holds at scale too. In three experiments with more than 4,000 participants, irrelevant moments of insight also made whole worldviews feel truer. The research's conclusion is practical: Any net-casting practice needs a cooling-off step between "wow" and "publish." ## Novelty has a shape What makes an idea genuinely new is not tone but structure. A classic study of "interesting" theories found they all reduce to one form: "what seems to be X is in reality non-X." The other reliable source is connecting fields that never talk. In 1986 a researcher noticed one literature linking fish oil to blood properties, and another linking a circulation disorder to exactly those properties — The two literatures never cited each other. His hypothesis was later supported by clinical trials. The sociologist behind the X-is-really-not-X pattern put the point bluntly: a theorist is remembered "not because his theories are true, but because they are interesting." Interest is a relationship with the audience: Up to a point, the more strongly an audience holds an assumption, the more interesting they find a proposition that denies it. Swanson's own definition of the second source is the best short description of the method: knowledge can be "public, yet undiscovered, if independently created fragments are logically related but never retrieved, brought together, and interpreted." He did it again with migraine and magnesium. ## A bridge in practice: what counts as work while waiting The research works through one example of a bridge, and it shows the pattern outside science. A court and a legislature looked at the same thing — a person with a ride-hailing app open, waiting for a ping — and filed it differently. Britain's Supreme Court held that drivers' working time "included any period when the driver was logged into the Uber app within the territory… and was ready and willing to accept trips." Ontario's law took the other view; it sets minimum pay "for each work assignment," which starts when a worker accepts an assignment and excludes travel before and after. The result: Logged-in waiting time is unpaid. The research names what that is: two common-law jurisdictions looked at the same phenomenon, a person with an app open waiting for a ping, and put it in opposite ontological categories: work in the UK, not-work in Ontario. It is the denied assumption, found in law rather than invented, and it is checkable in the documents. ## What pushes back against the pull The research lists countermeasures, each aimed at one of the failures above. The first is to ask for spread rather than a single answer; one method asks for several responses with probabilities, and its authors report a 1.6–2.1× gain in diversity for creative writing. The second is to name the obvious views before asking for anything else, then ask only for coherent claims none of those groups is positioned to make. The third deals with the drift: Use fresh sessions for each lure, run at least two model families, and never refine one idea for more than a few turns in the same thread. The last keeps the model out of its own verdict: Grade catches yourself, or with a fresh session given only the rubric and not the conversation. The research is careful about what it is offering: Treat the four-test rubric and the workflow as my synthesis, not validated instruments. ## What this explains It explains why AI brainstorming so often produces ideas that feel fresh and turn out familiar: the model is built to give the likeliest answer, and the likeliest answer is what others already think. It explains why checking with a second model, or refining for an hour, makes an idea more confident without making it more original. And it points to what works instead: ask for the unlikely answers on purpose, name the assumption you want overturned, bring in material the model has not seen, and connect fields that do not cite each other. The machine can search; deciding where to look is still the work.