The AI-Slop Trap: Human-in-the-Loop Editing and Hallucination as a Brand Risk

Let's run the news from the week of June 9th through the 16th, 2026, and there are three things worth your time.

First, OpenAI gave everyone hands-on memory controls. On June 12th, an update rolled out to all web users, and now you can delete individual memories from your memory summary page, edit those summaries by typing or highlighting text, and turn memory off entirely with a "delete and turn off memory" option in the three-dot menu. This sits on top of the "Dreaming V3" memory architecture from June 4th and 5th, which auto-synthesizes memory across your past conversations instead of a hand-curated list. Plus and Pro users also got roughly twice the capacity. Why you care: ChatGPT now quietly builds a persistent profile from your chats, so audit what it keeps to keep brand-voice and project context clean between clients. Go into Settings, Personalization, Memory, and prune anything stale before your next big drafting session.

Second, Google made information agents in AI Mode broadly available. On June 12th, Google's search VP of product, Robby Stein, said they're now live in all AI Mode languages and markets for Google AI Ultra subscribers, with wider access coming this summer. Introduced at Google I/O on May 19th, an information agent is a persistent, autonomous watcher. It monitors the open web, news, social, and real-time data around the clock and sends synthesized updates with links when something relevant surfaces. You write the request in plain language, like "track this competitor's launch," instead of exact-keyword alerts. Think of it as a smarter Google Alerts, gated behind the AI Ultra tier for now. If you have it, point one at a competitor and test it against your current alerts.

Third, CallRail says it's the first SMB and agency-focused provider to offer ChatGPT Ads attribution. Announced June 10th, it automatically attributes calls, texts, and form submissions to specific ChatGPT campaigns and ad groups, shows them next to Google, Meta, and Microsoft, and sends conversion data back to OpenAI to improve bidding. Setup is self-serve, an API key plus a conversion pixel ID, at no extra cost to eligible customers. They cite a reportedly vendor-sourced figure that 37 percent of consumers now start searches with AI. ChatGPT Ads have been hard to measure since US testing began in January, so this closes the loop.

In the background, OpenAI launched a Partner Network around June 14th with a reported $150 million behind it, launch partners like BCG and Accenture, and a goal of certifying 300,000 consultants by year-end. Mostly relevant for enterprise buyers.

And a quick AI-search visibility snapshot, fast-moving, not breaking. No major new study landed this window, so the freshest numbers are trailing. Google AI Mode reportedly ends about 93 percent of sessions without a click, at roughly 75 million daily users. AI Overviews coverage climbed from about 34 and a half percent of queries in December to about 48 percent by March, ending without a click around 83 percent of the time. On referrals, ChatGPT reportedly drives about 62.6 percent of measurable B2B AI referrals, Claude about 18.5, Gemini about 10.6, and Perplexity about 7.3, though Perplexity cites far more sources per answer. Trackers like Semrush One, Profound, and Evertune are worth checking.

Today we're talking about the AI-slop trap, and it has two halves that most marketers treat as one problem when they're really two. The first half is that AI writing has a sound, a generic, machine-made sound that your audience can now hear, and once they hear it, they trust you less. The second half is harder and more dangerous. AI doesn't just write blandly, it invents facts, confidently, and when you publish those invented facts under your brand, you own them. Legally. We've named the slop tell before, back when we built your brand-voice profile with style guides and few-shot examples. Today we go deeper, and we end with a copyable two-pass edit workflow you can run on every draft.

I want to be clear about the stakes before we get into mechanics, because this isn't a style complaint. The sound problem and the fact problem are both trust problems, just on different clocks. The generic sound leaks trust slowly. A reader who can tell a robot wrote your email trusts your next email a little less. The fabricated fact breaks trust all at once, and sometimes it breaks the law along with it. Both come out of the same machine, and both get caught by the same thing, a human who actually reads the draft before it ships.

Let's start with what AI slop actually is. The term emerged across 2024 and 2025 to describe low-effort, mass-produced, generically-AI-sounding content. Think of it as the textual equivalent of spam. And here's the most useful distinction, one that Charlie Guo draws in his "Field Guide to AI Slop," and that a write-up from Paralect echoes. There's a vocabulary tell and a structural tell, and the structural one is bigger. When a reader says "this sounds like ChatGPT," they're usually reacting to structure, the rhythm, the constant hedging, the listicle sameness, the total absence of a point of view, as much as to any specific word. So if you only hunt for banned words, you'll miss most of what gives you away.

Why do models produce this in the first place? It's baked into how they work. Large language models are trained on enormous amounts of formal, published writing, which already skews formal, hedged, and cautious. And when the model generates text, it's picking the statistically safest next word, the one that appeared most often and offends no one. Colin Gorrie's piece on why ChatGPT writes the way it does, and the analysis from Walter Writes, both land on the same point. The output drifts toward a small set of words that sound "well-written" and toward reciting consensus with no stance of its own. Safe is the whole design goal, and safe is exactly what reads as machine-made.

There's a second-order effect here worth sitting with. As more and more published text is itself AI-written, the next generation of models trains on that output, and the tells get reinforced. The bland average gets blander. So the sound you're learning to recognize isn't a fixed target, it's drifting, and the words that scream AI today are partly the residue of yesterday's AI writing getting absorbed back into the training data. That's exactly why a human pass, with a real point of view, is the thing that breaks the loop.

Now the vocabulary, because you do want a kill list, you just shouldn't stop there. The words flagged over and over are: delve, tapestry, landscape, leverage, multifaceted, comprehensive, furthermore, crucial, utilize, robust, navigate, elevate, intricate, meticulously, synergy, empower, ecosystem, underscore, seamless, game-changer, pivotal, foster, realm, testament, beacon, and nuanced. If you've read any AI draft, you've seen a cluster of these. And this isn't just vibes. A Max Planck Institute analysis found that words like delve, robust, and pivotal spiked by more than 50 percent in published text after ChatGPT's release. So the published corpus itself is now visibly contaminated, which is part of why these words ring the bell so loudly. There's a fun bit of color tied to delve specifically. Its surge has been linked to the human labeling work behind reinforcement learning from human feedback, RLHF for short, which was done in regions where delve is common in formal English. That's plausible, not proven, but it's a nice reminder that these tics have human fingerprints on them.

Then there are the phrases, the formula openers and closers. Margaret Efron's piece on the obvious tells lists the usual suspects: "it's important to note that," "in today's fast-paced world," "when it comes to," "let's dive into" or "dive deep into," "navigating the landscape of," "in conclusion," "in summary," "it's worth noting," "whether you're this or that," and "at the end of the day." When you see two or three of these in one piece, you're looking at an unedited draft.

I want to single out one structure because it has become the loudest tell of all. It's the contrastive negation frame, the "it's not just X, it's Y" move. "It's not just a tool, it's a partner." "This isn't about features, it's about outcomes." That used to be a perfectly good rhetorical device. Writers used it deliberately, sparingly, for emphasis. But the models over-deploy it so relentlessly that it now screams "I used ChatGPT." Ole Reissmann wrote a whole piece on how contrastive negation went from technique to tell. And notably, Wikipedia's community guide, "Signs of AI Writing," explicitly calls out the "it's not X, it's Y" structure as a flag. When your encyclopedia's volunteer editors have a written rule against a sentence shape, that shape is burned. So in this episode, you'll notice I'm not using it, on purpose.

The em-dash is its own little saga. ChatGPT over-uses the em dash because, statistically, em dashes show up in prose that got labeled as well-written, so the model reaches for them constantly. But here's the nuance, and Rolling Stone covered this well. The em dash is not a reliable detector on its own. Plenty of strong human writers, present company included, lean on them heavily, and other models use them far less. So an em dash alone proves nothing. It's only a signal in combination with everything else. For what it's worth, I'm narrating this script without any, partly to make the point.

And then the rest of the structural tells, which matter more than any single word. Hedging everywhere, the constant "can," "may," and "often" that drains all conviction. Listicle sameness, where every item is the same length and the same shape, like a row of identical fence posts. Consensus-recital with no point of view, the piece that summarizes what everyone already agrees on and risks nothing. The relentless tricolon, the "fast, scalable, and reliable" three-beat rhythm, over and over. The tidy summarizing conclusion that adds nothing you didn't already read. Over-signposting, "first, second, finally," when the structure was already obvious. And empty enthusiasm, "in the ever-evolving world of." Paralect and Louis Bouchard both catalog these, and once you can name them, you can't unsee them.

Here's one of those structures caught in the act. An AI draft of a product email might open with, "in today's competitive market, businesses are constantly seeking innovative solutions to streamline their workflows and drive meaningful results." Read it back. It says almost nothing. Every noun is generic, there's no specific reader, no specific product, no claim anyone could disagree with. A human rewrite might be, "your team is losing an hour a day to manual data entry, and here's the one setting that stops it." Shorter, concrete, and it picks a fight with a real problem. The slop version offends no one and moves no one. That's the tell in miniature.

So that's slop, the sound problem. Now let's get to the part that should genuinely scare you, because the sound problem costs you trust slowly, while the fact problem can cost you money and a lawsuit fast. Hallucination is when the model states something false with total confidence, and when you publish it, you, not the model, are on the hook.

Start with the cleanest precedent, Moffatt versus Air Canada, decided by the British Columbia Civil Resolution Tribunal in February 2024. Air Canada's website chatbot told a customer he could apply retroactively for a bereavement fare. The actual policy didn't allow that. Air Canada actually argued in front of the tribunal that the chatbot was a separate legal entity responsible for its own actions. The tribunal rejected that flatly, found negligent misrepresentation, and held the company liable for everything on its site, including what the bot said. The damages were small, around 812 Canadian dollars plus fees, but the principle is enormous. You own what your AI says to your customers. There is no "the bot did it" defense.

Then there's the wave of lawyers getting sanctioned for fake AI citations, and this is instructive because lawyers are professional fact-checkers and they still got burned. The first famous one was Mata versus Avianca, in the Southern District of New York, where Judge Castel issued sanctions in June 2023. Two attorneys, Steven Schwartz and Peter LoDuca, filed a brief full of fabricated ChatGPT cases, fake quotations, invented citations, the works. They were hit with a 5,000 dollar joint sanction and ordered to send corrective letters to every real judge who'd been falsely named in the fake cases. Embarrassing doesn't begin to cover it.

You might tell yourself that was an early-days, solo-practitioner mistake. It wasn't a one-off. In February 2025, in Wadsworth versus Walmart, in the District of Wyoming, a major national firm, Morgan and Morgan, filed a brief where its own in-house tool, called MX2.law, hallucinated eight of the nine cited cases. The lead attorney, Rudwin Ayala, lost his pro hac vice admission, his permission to appear in that case, and was fined 3,000 dollars. Two other attorneys were fined 1,000 each, 5,000 total. Morgan and Morgan is roughly the 42nd largest US firm by headcount. This is not a fringe problem.

And even the AI companies and the experts who study this aren't immune, which I find almost reassuring, because it means nobody gets a pass. In Concord Music Group versus Anthropic, in the Northern District of California in May 2025, Anthropic, the maker of Claude, admitted that one of its own expert's declarations contained a citation where Claude had fabricated the title and the authors of a real article. The judge struck the paragraph. And a Stanford misinformation professor, Jeff Hancock, filed an expert declaration defending a deepfake law, and that declaration itself contained AI-fabricated citations. The court excluded his testimony. So the person testifying against AI misinformation got tripped by AI misinformation.

The penalties have been climbing, too. Through 2025 and into 2026, sanctions went from gentle warnings, to a thousand to five thousand dollars, to ten thousand and up, with reported instances as high as about 86,000 dollars. In February 2026, the Ninth Circuit sanctioned two attorneys 2,500 dollars each and suspended them for fabricated citations. There's a running tracker, Damien Charlotin's database, that has reportedly logged well over a thousand documented cases of AI hallucinations in legal filings, more than one new decision per day. Treat that exact count as fast-moving, because it climbs constantly.

And the sanctions aren't all small or all the same. Beyond Avianca and Morgan and Morgan, there's a string of them through 2025. In a case called Mid Central Operating Engineers versus Hoosiervac, a court imposed 6,000 dollars, reduced from a recommended 15,000, over three briefs with fabricated citations. In Tercero versus Sacramento Logistics, an attorney was fined 1,500 dollars, and the penalty was worse because the attorney falsely denied using AI. That's a pattern worth flagging, the cover-up draws more punishment than the mistake. And some penalties aren't about money at all. In one Georgia bankruptcy matter, a filer was hit with a five-year requirement to disclose AI use on every future filing. In an Alabama case, Johnson versus Dunn, an attorney was disqualified and referred to the bar. In a Florida case, Dubinin versus Papazian, the case was dismissed outright, with fees and a bar referral. The throughline is that the consequences have moved well past a slap on the wrist.

Now bring this back to marketing, because the same liability principle reaches you through the Federal Trade Commission, the FTC. In September 2024 the FTC launched Operation AI Comply, five enforcement actions against deceptive AI claims, the thing people call "AI washing." Targets included DoNotPay, which had billed itself as "the world's first robot lawyer," and Rytr, for generating fake reviews. And this carried straight into 2025 under the new administration, so it's not a one-administration whim. The core principle the FTC states is that it will aggressively pursue people and companies that make false or unsubstantiated claims about an AI product, its capabilities, or its results. Here's the part marketers miss. Publishing an AI-fabricated claim of fact, a made-up stat, a phony efficacy figure, is exactly the kind of false or unsubstantiated claim the FTC Act already reaches. You don't have to be selling AI to get caught. You just have to publish a number you can't back up.

And there's a perfect irony buried in the FTC's enforcement. In a final order in August 2025, the FTC went after Workado, which marketed its AI Content Detector as 98 percent accurate. That figure came from a study on academic text. When the detector was tested on general content, it came in around 53 percent accurate, barely better than flipping a coin. The FTC required Workado to stop making the claim. So that case does double duty. It's an enforcement example, and it's hard proof that AI detectors don't work, which we'll come back to.

Let's stay on brand and publisher incidents, because these are the ones that hit trust directly. In January 2023, CNET quietly published more than 70 AI-generated finance articles. After readers caught factual errors, including a botched explanation of compound interest, CNET had to issue corrections on a large share of them. In November 2023, Sports Illustrated published product reviews under entirely fake AI-generated author personas, complete with AI-generated headshots. One was named "Drew Ortiz." When journalists started asking questions, the bios vanished. It was a major trust blow and there was executive fallout. And the bigger names aren't safe either. In 2025, Bloomberg rolled out AI-generated article summaries and had to issue dozens of corrections. And just recently, in June 2026, KPMG, one of the big professional-services firms, pulled a report on AI usage after the tool GPTZero flagged inaccuracies consistent with AI hallucination. A firm that advises the world on AI tripped on its own AI-about-AI report. Nobody is too sophisticated for this.

The fabricated-citation problem reaches into published books and research, too. Springer Nature retracted a textbook called "Mastering Machine Learning: From Basics to Advanced" after 25 of 46 sampled references couldn't be verified, more than half. And the scale is getting worse, fast. A Lancet-published audit in 2026 found that papers with fabricated references rose from one in 2,828 in 2023, to one in 458 in 2025, and reached one in 277 in early 2026. A separate study indexed in PubMed Central found that roughly one in five AI-generated references were entirely fabricated. And here's the mechanism, which you need to internalize, because it explains why this keeps happening. Models generate citations by statistical inference, not by looking anything up in a database. So a plausible-looking author, plus a plausible journal, plus a plausible year, plus a plausible DOI, that digital object identifier string that points to a paper, is precisely the kind of thing the model invents. It's not retrieving a real reference and getting it wrong. It's pattern-matching what a citation should look like and printing one that doesn't exist.

And it's not only lawyers, and not only journals. One breakdown of the 2025 legal data found that people representing themselves in court, with no lawyer, actually generated more of these incidents than licensed attorneys did, 304 versus 219. That tells you how naturally people trust a confident-sounding answer when they don't have a trained skeptic in the room. The research world has the same problem at the top. When the tool GPTZero scanned the accepted papers at NeurIPS 2025, one of the flagship AI conferences, it found at least 100 hallucinated citations across 51 papers, authors and papers and identifiers that don't exist, and they slipped past expert peer review. So the people building AI are publishing AI's fabrications about AI. If they can miss it, so can you, and so can your agency.

So how often does this actually bite working marketers? We have good data here, from NP Digital's "AI Hallucinations and Accuracy Report," published in February 2026. They surveyed 565 US digital marketers and tested 600 prompts across ChatGPT, Claude, and Gemini. The findings are sobering. About 47 percent of marketers encounter AI inaccuracies several times a week. And 36 and a half percent admit that hallucinated or incorrect AI content has already been published publicly, on their watch. More than 70 percent spend one to five hours a week fact-checking AI output, which tells you the careful ones already know. But 23 percent say they're comfortable using AI output without any human review at all, and that's the dangerous slice, the people most likely to ship a fabrication. Nearly 78 percent accept some level of AI inaccuracy because, to them, speed outweighs the cleanup cost.

The error types in that report are worth naming, because they tell you where to look. The most common are omissions, where the model leaves out something crucial, outdated information, outright fabrication, and misclassification, getting a category wrong. And almost all of it is delivered with high confidence. That confidence is the trap, and I want to hammer this. The best-performing model in their test was still only about 59.7 percent fully correct, and every model faltered on multi-part questions, niche topics, and anything requiring real-time information. So even your best tool is wrong roughly 40 percent of the time on hard questions, and it never sounds wrong. There's a counterpoint worth holding alongside this, from a 2025 IBM study, which found that more than 76 percent of enterprises now build a human-in-the-loop review step specifically to catch fabrications before deployment. The grown-ups have already figured out that the human pass is mandatory.

Sit with that 36 and a half percent for a second, because it's the number that should change your behavior. It means more than a third of marketers, people doing this for a living, have already let a hallucination reach the public under a brand they're responsible for. Not "might someday." Already did. And the 23 percent who skip human review entirely are the supply line for the next batch. The lesson isn't that AI is too dangerous to use. Plenty of careful people are using it every day and shipping clean work. The lesson is that the careful ones built a review step, and that step is not optional. It's the price of using the tool at all.

Now let's deal with the detector fantasy directly, because a lot of marketers want a button that says "this is AI" or "this is true," and that button does not exist. OpenAI killed its own detector. Its AI Text Classifier was discontinued on July 20th, 2023, and OpenAI's stated reason was its low rate of accuracy. The company that makes the model couldn't reliably detect the model. A Stanford study found that seven different detectors flagged TOEFL essays written by non-native English speakers as AI-generated about 61 percent of the time, sometimes unanimously, while almost never misflagging native writers. So detectors don't just fail, they fail with a bias that would get you into a discrimination problem if you used them on real people. And remember the Workado case, the 98 percent claim that tested at 53 percent on general content. Detectors are also trivially evaded. A little light paraphrasing, or running a draft through a "humanizer" tool, defeats them, while clean, careful human writing gets falsely flagged. The deepest issue is conceptual. A detector, at best, makes a bad guess about who typed something. It can't tell you whether a stat is true. The only reliable check is human judgment plus checking the primary source.

And I get why the button is so tempting. You're busy, you're shipping volume, and a green checkmark that says "human-written, all true" would let you skip the slow part. But that checkmark would be lying to you twice. Once about who wrote it, which doesn't matter, and once by implying the facts are sound, which it has no way of knowing. The detector industry is selling the feeling of safety. The actual safety is boring and manual, and it's the only thing that works.

Which brings us to the payoff, the thing you can actually run on every draft. It's a two-pass human-in-the-loop workflow. Pass A is de-slop and voice. Pass B is fact-check and claims. Do them in that order, voice first, facts second, because they use different parts of your brain and mixing them makes you sloppy at both. This is the human-edit idea we keep coming back to, now turned into a checklist.

Here's Pass A, restoring a human point of view, drawn largely from Louis Bouchard's guide to editing AI drafts. Step one, find and kill the tell vocabulary. Do a literal search for delve, leverage, robust, tapestry, landscape, seamless, and game-changer, and the formula phrases, "it's important to note," "in today's fast-paced world," "let's dive in," "in conclusion." Delete or replace every one. Step two, break the "it's not X, it's Y" frames and the relentless tricolons, and vary your sentence length, because uniform rhythm is itself a tell. Step three, and this is the one that earns your byline, cut the hedging where you actually have a stance, and then add the stance. Put in the point of view, the opinion, the specific anecdote, or the contrarian take that the AI structurally cannot supply, because it was trained to offend no one. Step four, replace every generic example with one real, specific, checkable example. Step five, read the whole thing aloud and ask one question, does this sound like a person who actually knows this audience? If it doesn't, you're not done.

Here's a quick before-and-after so this isn't abstract. The AI draft says: "In today's fast-paced digital landscape, leveraging AI can be a game-changer for businesses looking to elevate their content strategy. It's not just about saving time, it's about unlocking growth." Run Pass A and you get something like: "Most marketers reach for AI to save time. The bigger win, if you edit it right, is that it frees you to do the strategic thinking you never have time for." Same idea, but now there's a stance, a real claim, and not a single tell word. That's the work.

Now Pass B, fact-checking every claim of fact, drawn from Articulate's guide to fact-checking AI content. Step one, highlight every load-bearing assertion, and I mean every number, name, date, quote, statistic, citation, URL, and claim of fact. If it could be true or false, it gets highlighted. Step two, verify each one against a primary or authoritative source, and open the original report or page yourself. Don't accept "according to a 2024 study" without finding the actual study. And here's the subtle trap, even when the model cites a real source, that source often doesn't actually say what the draft claims. So you're confirming two things, that the source exists, and that it supports the point. Step three, click every link. The model fabricates plausible URLs and DOIs, so a 404 or a redirect to something unrelated is a giant red flag. Verify product specs and prices against the vendor's own page, not against what the model remembers. Step four, second-source anything high-stakes, anything legal, medical, financial, or competitive. Step five, the gate I want you to tape to your monitor: "would I put my name on this in front of the FTC?" If you can't point to where a claim came from, you cut it or you caveat it. And for high-impact pieces, a named human approver signs off, a real person whose name is on it. Step six, keep a source trail for anything high-risk, so when someone asks "where did this number come from," you have an answer.

Let me make Pass B concrete with one example. Say your AI draft hands you this sentence: "according to a 2024 Gartner study, 73 percent of consumers trust brands that use AI transparency labels." It reads great. It's specific, it has a year, a named research firm, a clean number. Now run the check. Search for that exact study. If Gartner never published it, you've caught a fabrication that would have shipped with a fake authority bolted on, which is worse than no citation at all. If a real Gartner report exists but the actual number is 53 percent, or it's about something slightly different, you've caught a misquote, and that's just as damaging the moment a competitor or a journalist checks. And if you genuinely can't find the source in a few minutes, the rule is simple. The claim comes out, or it gets softened to something you can stand behind in public. A made-up statistic with a real company's name on it is exactly the kind of thing that turns into a correction, or a legal letter.

Let me leave you with the framings that make this stick, because I want you triaging, not panicking. The first is the triage line itself. AI is genuinely low-risk for creative and voice and structure work, ideation, rephrasing, outlining, generating tone variants, because there's no true answer there for it to fabricate. The risk lives entirely in claims of fact, stats, names, dates, quotes, legal and medical and financial assertions, specs, and citations. So you don't fact-check everything equally. You draw that line and you put your scrutiny on one side of it.

Let me put a finer point on what lands on the dangerous side of that line, because it's broader than just statistics. It's any proper noun the AI didn't get from you, a person's name, a company, a product, a job title. It's any date or dollar figure. It's any quote, because the model will happily invent a plausible sentence and attribute it to a real person. It's any claim about what a law, a study, or a competitor says. And it's any specification, a dimension, a price, an ingredient, a compatibility claim, because the model will invent specs you never gave it. If a sentence asserts something a customer could act on and be wrong about, it belongs on the fact-check side. The second framing is that confidence is the trap. The more polished and fluent a passage reads, the better a fake stat hides inside it. Polish is not accuracy, and your gut will keep telling you the smooth paragraph is the trustworthy one. It isn't. The third framing is a cost comparison. On one side, a one-to-five-hour-a-week fact-check pass. On the other side, a struck declaration, an excluded expert, five thousand dollars in sanctions plus a revoked court admission plus a national news cycle, a retracted textbook, a pulled flagship report, FTC exposure, or being held liable for what your chatbot promised a customer. The edit pass is the cheapest insurance in your whole stack. The fourth framing, and say it with me, the detector is not the safety net, the human is. OpenAI couldn't build a reliable detector, and the FTC fined a company for claiming it had one. The only repeatable safeguard is a person running the fact-check pass against primary sources. And the last one, the reason all of this matters, is brand trust compounding. The moment your audience catches one fabricated stat or one fake byline, they quietly discount everything else you've ever published. The slop tell erodes trust slowly and the hallucination erodes it fast, and trust is the actual asset you're protecting. That's what the two-pass edit is really defending.