Language models and idea diversity: homogenisation within a session, across users, and between companies

You ask an AI for a fresh angle on a problem. It gives you something that sounds sharp. You ask another model to check, and it agrees. You keep refining together for an hour, and the idea gets smoother and more confident. The research behind this article says each of those steps moves you toward the average, not away from it.

1Different models give much the same answer

A second model feels like a second opinion. The largest test of this, over 70 models and 26,000 open questions, found "intra-model repetition" and, "more so," "inter-model homogeneity, where different models produce strikingly similar outputs."Key findings So switching to another model does not reliably give you a second opinion.Key findings Part of the reason is training itself: The model has been trained to hand you the most typical answer.§3

2AI ideas look novel, then shrink when built

The most careful test of AI research ideas ran in two stages. In the first, with more than 100 researchers, reviewers judged model-generated ideas "more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility."Key findings Then the ideas were built. In the follow-up, 43 experts each spent over 100 hours executing randomly assigned ideas.Key findings Afterwards the model ideas' scores fell further than the human ones "on all evaluation metrics (novelty, excitement, effectiveness, and overall; p < 0.05),"Key findings and "for many metrics there is a flip in rankings where human ideas score higher than LLM ideas"Key findings. The research calls this the empirical version of "plausible-sounding but hollow synthesis."Key findings

The models are also poor judges of their own work. Si et al. found LLMs were unreliable at judging their own ideas.Details So asking the model whether its idea is original is the check least likely to help.

3Long sessions pull toward the model's favourite themes

The longer you iterate with one model, the narrower it gets. Asked for new combinations of ideas, one leading model reused the same few concepts almost every time: its ten most-used concepts appeared in 95.7% of outputsKey findings. Over a run of research reports, its topics drifted toward one subject it had not started on.

Other models showed the same pull to a lesser degree. For a second leading model the figure was 76.7%, versus 34.3% for a sampler that deliberately avoided "cognitively available" combinationsKey findings. The drift over a long run was measurable: Claude's reports drifted from 2 of 10 on interpretability to 8 of 10, even though the ideas had not started there.Key findings The research flags the source's limits: the paper is a recent preprint, and its model-specific numbers describe particular model versions on one research corpusCaveats.

4Surprising ideas come from outsiders, and look wrong at first

If the model drifts to the centre, the useful question is where surprising ideas usually come from. One study found that surprising combinations of research content and context are associated with higher impactKey findings, and that they tend to come from scientists crossing in from distant disciplinesKey findings. Another found that the papers a model finds most perplexing have bimodal fates: they are over-represented among the most celebrated work and also among the most dismissedKey findings. The research draws the uncomfortable lesson: Radical catches will often look wrong at first.Key findings

The same idea can be turned into a tool. By modelling which people are likely to make which inferences, researchers improved AI prediction of future discoveries by up to 400%Details, and by reversing the model to "avoid the crowd," produced promising "alien" hypotheses that people were unlikely to pursueDetails. The test that follows has two parts: score ideas on two separate axes, coherence (does it hang together?) and availability (is anyone already positioned to say it?)Details

5It narrows the crowd, not only the person

The effect is worse in aggregate. Writers given AI story ideas wrote more creative stories — and more similar ones. The researchers describe "an increase in individual creativity at the risk of losing collective novelty."§3 If everyone in a field prompts the same few models, the field converges, and each person feels more original while it happens.

6Novelty has a shape

What makes an idea genuinely new is not tone but structure. A classic study of "interesting" theories found they all reduce to one form: "what seems to be X is in reality non-X."Key findings The other reliable source is connecting fields that never talk. In 1986 a researcher noticed one literature linking fish oil to blood properties, and another linking a circulation disorder to exactly those properties — The two literatures never cited each other.Key findings His hypothesis was later supported by clinical trials.

The sociologist behind the X-is-really-not-X pattern put the point bluntly: a theorist is remembered "not because his theories are true, but because they are interesting."Key findings Interest is a relationship with the audience: Up to a point, the more strongly an audience holds an assumption, the more interesting they find a proposition that denies it.Key findings Swanson's own definition of the second source is the best short description of the method: knowledge can be "public, yet undiscovered, if independently created fragments are logically related but never retrieved, brought together, and interpreted."Key findings He did it again with migraine and magnesiumKey findings.

7A bridge in practice: what counts as work while waiting

The research works through one example of a bridge, and it shows the pattern outside science. A court and a legislature looked at the same thing — a person with a ride-hailing app open, waiting for a ping — and filed it differently. Britain's Supreme Court held that drivers' working time "included any period when the driver was logged into the Uber app within the territory… and was ready and willing to accept trips."Details Ontario's law took the other view; it sets minimum pay "for each work assignment," which starts when a worker accepts an assignment and excludes travel before and afterDetails. The result: Logged-in waiting time is unpaid.Details

The research names what that is: two common-law jurisdictions looked at the same phenomenon, a person with an app open waiting for a ping, and put it in opposite ontological categories: work in the UK, not-work in Ontario.Details It is the denied assumption, found in law rather than invented, and it is checkable in the documents.

8What pushes back against the pull

The research lists countermeasures, each aimed at one of the failures above. The first is to ask for spread rather than a single answer; one method asks for several responses with probabilities, and its authors report a 1.6–2.1× gain in diversity for creative writingDetails. The second is to name the obvious views before asking for anything else, then ask only for coherent claims none of those groups is positioned to makeDetails. The third deals with the drift: Use fresh sessions for each lure, run at least two model families, and never refine one idea for more than a few turns in the same thread.Details

The last keeps the model out of its own verdict: Grade catches yourself, or with a fresh session given only the rubric and not the conversation.Details The research is careful about what it is offering: Treat the four-test rubric and the workflow as my synthesis, not validated instruments.Caveats

9What this explains

It explains why AI brainstorming so often produces ideas that feel fresh and turn out familiar: the model is built to give the likeliest answer, and the likeliest answer is what others already think. It explains why checking with a second model, or refining for an hour, makes an idea more confident without making it more original. And it points to what works instead: ask for the unlikely answers on purpose, name the assumption you want overturned, bring in material the model has not seen, and connect fields that do not cite each other. The machine can search; deciding where to look is still the work.