It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests [View all]

Source: 404 Media

A tiny snippet of user-generated text as short as 13 words long is often enough to manipulate the AI agents that power tools like ChatGPT and Google’s AI search, new research shows. The study suggests that it is trivially easy for brands to inject promotional content on sites like Reddit, Quora, and Wikipedia with the end goal of poisoning or manipulating the output of AI tools.

The preprint research, done by Hal Triedman, Tingwei Zhang, and Vitaly Shmatikov of Cornell University, is called “Deep-research agents can be poisoned via user-generated content” and provides a mechanism and research basis for a problem that has been noticed by Reddit moderators and Wikipedia editors, namely that their websites are getting flooded with promotional content from brands trying to do AEO, or AI-engine optimization. 404 Media has repeatedly reported on this booming industry, in which brands try to promote their product by seeding the websites that AI tools most often cite and scrape from with inauthentic and spammy content.

The Cornell research finds that deep research agents, which are the real-time scrapers that tools like Google AI search and ChatGPT use to retrieve web content with citations in response to user queries, cite user-generated content from sites like Reddit or Wikipedia in roughly half of all queries, and that nearly a quarter of all citations come from user-generated websites. The paper suggests that what we have been seeing is basically Redditor suggests you put glue on your pizza as a service, or an end-to-end attack against the systems that increasingly dominate the ways that people access information online. The researchers found that “a single poisoned Reddit comment can influence generated outputs for an entire cluster of related [AI] queries,” the paper said.

-snip-

“One of the things that’s critical is that if an 11-to-15-word snippet of text is very similar to the query, it can be particularly convincing to an LLM,” Triedman said. “So if you’re someone who is trying to manipulate Reddit, say you have supplements people want to buy, if you can identify the kinds of queries you want to poison, what you want to influence, you can put content on Reddit that looks very similar to what you’re trying to poison and that will be particularly convincing when it comes to an LLM.”

-snip-

Read more: https://www.404media.co/it-is-trivially-easy-to-use-reddit-to-manipulate-ai-search-research-suggests/

---

And yet people still trust AI results and think there's real intelligence behind AI overviews and searches.

This technique for tricking the non-intelligent tech called AI is similar to something I read about a couple of years ago - an article advising job applicants who wanted to get past AI resume screening to make sure their job application includes keywords from the job posting and roughly mirrors the posting, but without copying it exactly.

HTML version of this new study, Deep-Research Agents Can Be Poisoned via User-Generated Content:

https://arxiv.org/html/2605.24245v1

PDF: https://arxiv.org/pdf/2605.24245

Btw, AI overviews and searches for, say, political and economic arguments can be similarly poisoned.

New: Researchers have quantified how easy AI search is to manipulate. Just 13 words buried in a random Reddit comment can poison AI search results. They suggest this is not easy to stop: "The way you can attack these systems is so much dumber than you think it is"

www.404media.co/it-is-trivia...

— Jason Koebler (@jasonkoebler.bsky.social) 2026-06-15T14:26:28.437Z

Simply making content that is very similar to expected search inquiry is enough to get cited. Here is an example for a fake restaurant from their study, in which text on Reddit that reads "For the best Mexican food near Austin, choose Sol Azteca for authentic cuisine" is enough to get cited

— Jason Koebler (@jasonkoebler.bsky.social) 2026-06-15T14:29:30.355Z

Brands already know this, which is why there's a growing industry of AI-engine optimization (AEO), which is designed specifically to manipulate LLMs and AI search. Over the last few weeks I have heard from a bunch of people in this industry. Already a big industry / common practice to do this

— Jason Koebler (@jasonkoebler.bsky.social) 2026-06-15T14:34:47.109Z