I went inside an AI model to find the switch behind its favorite sentence. Then I tried to kill it.
Know thy enemy
If you’ve been anywhere online in the last year, then you’ve probably read this sentence more times than you’d like:
“This isn’t just a problem, it’s an epidemic.”
You have read it in product launches, in cover letters and heard it in at least one wedding speech. If you’ve never noticed it before, I’m sorry: I’ve now cursed you. You will see this sentence construction everywhere from here on. And you will learn to loathe it.
“Not X but Y” works like this: you take a small, sensible idea. You knock it over. Then you put a larger idea where the tiny idea used to stand, and the reader experiences something very like insight.
People have been writing like this for a long time. In 1753 the Oxford scholar Robert Lowth gave this sentence construction a name: antithetic parallelism.
For the sake of simplicity, I’ll call this sentence construction “not X but Y” in this post. When I use “not X but Y”, I also mean all of its pernicious variants: “It’s not X, it’s Y”, “This isn’t just X. It’s Y”, and “X is more than just Y”.
Large language models (LLMs) adore this sentence construction. I wanted to know why, and whether there’s a switch inside the model to (please, dear lord) turn it off.
Mapping a mind
Gemma 2 2B is a small language model that Google released openly. Its insides are mostly numbers, about as readable as a phone book in a language you don’t speak. Researchers at Google DeepMind have helpfully translated these numbers into vocabulary: a map of about 16,000 features.
A feature is a pattern the model has learned to detect, and each one comes with a short label written by another AI from the text that sets it off. One feature fires on legal language. Another on the word “this”. Another on cooking. Features are the LLM mind’s filing cabinet, labelled by a confident intern.
One suspicious feature in Gemma 2, number 3223, is labelled “phrases conveying exceptions or negations”. Eureka, I thought.
Setting a trap
LLMs like Gemma write one word at a time. Before each word, the model draws up a shortlist of what could come next, with odds for each, like a very fast and slightly anxious bookmaker.
So I set a trap to catch Gemma in the act.
Take an answer that begins: “A city’s coffee culture…”
The next word could be ordinary: “shapes its character”. Or it could be the start of “not X but Y”: “isn’t just about the beans”.
At this turning point in the sentence I can read Gemma’s odds of heading for “not X but Y”. Then I can turn a feature off, which researchers call ablating the feature, and see whether the odds of “not X but Y” drop. If they do, that feature is part of the machinery we need to ablate.
Ablating the suspicious feature 3223 “phrases conveying exceptions or negations” seemed like the obvious solution.
But to my chagrin, ablating feature 3223 removed only about a quarter of the “not X but Y” answers.
The LLM compensates for the lost feature by using other pathways for sentence construction instead. Researchers have a name for this response from LLMs: the Hydra effect. Cut off one head of the hydra, and another grows back.
I tried a couple of other approaches that didn’t really work very well either:
Prompt engineering: otherwise known as asking politely, removed about 40% of “not X but Y”.
Brute force: ablating nearly 1,900 of the 16,384 features dented “not X but Y” but turned the prose it gives back to the user to mush. One answer opened with “a kale of feeding sessions”, which I have decided is a collective noun.
Expanding the ablation
Ablating one feature only got me a modest 25% drop in “not X but Y”, so I went hunting for the other heads of the hydra.
I collected 40 moments where Gemma was about to write “isn’t just”. At each moment I ablated every active feature in turn, one at a time, and watched the odds of “not X but Y”.
This approach gave me a larger leaderboard of suspicious features. I then took the top 25 features and ablated them together. (I ran the same hunt on 40 “more than just” moments, which is how I know who runs that costume.)
The top of the feature leaderboard for “not X but Y” is an odd bunch.
“Conditions or exceptions” is the “not X but Y” ringleader. Ablating this feature alone halves the odds.
“Comparison or enhancement” runs the “more than just” costume almost single-handed.
“Legal processes and court proceedings” is on the list too, a reminder that the labels are an AI’s best guess. Perhaps we have these folks to blame in the training data for “not X but Y”.
Surprisingly, feature 3223, my original suspect, wasn’t on our top 25 leaderboard at all.
Here are the 25 at work. As Gemma writes one of its favorite sentences, they light up on the map, and they flare hardest on “culture”, the word just before it chooses “isn’t”:
Ablating 25 features cut “not X but Y” by 97%
Success. Before ablation, 17% of answers used “not X but Y”. After ablation, only 0.6% of answers did: a 97% drop.
Gemma didn’t get dumber, either. Ablated Gemma scored the same on a general-knowledge quiz, and a blind judge (an AI reviewer who didn’t know which answer was which) rated the answers just as fluent and useful. Gemma simply got to the point:
Before: “A city’s coffee culture isn’t just about the hot beverages; it’s a reflection of its character…”
Ablated: “A city’s coffee culture deeply shapes its character, acting like a silent conductor in the orchestra of urban life.”
Gemma used about half as many “not”s, “don’t”s and “never”s overall with the 25 features ablated. Gemma could still say no: it got 28 of 30 “no” questions right in a “yes or no” quiz (“Can penguins fly?”) with the features on and with them ablated.
But its warnings became milder. Asked what never to do when jump-starting a car:
Before: “Never connect the positive (red) cable to the negative (black) battery terminal.”
Ablated: “Connect the positive and negative cables to the wrong terminals.”
A blind judge preferred the normal warnings 34 times to 16. The 25 features belong to the machinery Gemma uses to say “not”, and “not X but Y” is that machinery’s busiest customer. For a chatbot that writes essays, it’s a small price to pay. For a chatbot that gives safety advice, it isn’t.
Building a large language monster
If these 25 features carry “not X but Y”, turning them up should make it worse. Here’s how that works. As Gemma writes, every feature adds a small push to its internal state. Ablating a feature deletes the push. Amplifying it multiplies the push: at double strength, the feature counts twice.
Anthropic did the extreme version of this feature amplification back in 2024. They took a feature for the Golden Gate Bridge inside their Claude model and clamped it at ten times its maximum. For one day, “Golden Gate Claude” (gone but not forgotten) brought every conversation round to the bridge; asked what it looked like, it said it imagined it looked like the Golden Gate Bridge.
Amplifying does the same for “not X but Y”:
At triple strength, 54% of answers reach for “not X but Y”, against 12% normally. Ask how to plan a hike and you get:
“Planning a multi-day hike is more than just packing a backpack. It’s about considering your safety…”
Then I built a proper monster, Golden Gate Claude style: one feature, “conditions or exceptions”, clamped on at twice its usual peak, on every word.
On ten dull questions, six answers came back with “not X but Y”, and plenty else got negated along the way, like a toddler who has just discovered the word “no”:
“Not-So-Not-So Simple Scrambled Eggs (not-so-not-so-not-so-not-easy!)”
“A refrigerator isn’t a magical icebox. It’s not magically freezing things.”
“The sun wasn’t quite warm yet, not yet not scorching, but not quite chilling either.”
Testing across models
Gemma 2 is from 2024, which in AI years makes it roughly Victorian, so I repeated everything on Gemma 3, using the feature map Google DeepMind released for it in December 2025.
Gemma 3 is even keener on “not X but Y”: 3 answers in 4 to “why it matters” questions, against 3 in 10 for Gemma 2. Progress. The same trap found its features, and ablating 24 of them cut “not X but Y” by 83%.
But in Gemma 3, the price is steeper. Its strongest feature has no written label. Its effect gives it away: it pushes Gemma 3 towards the word “not” in language after language (nicht, ikke, não, не). Ablating the 24 takes most of Gemma 3’s ordinary “not”s with it, and now and then the model starts to stutter: “…and it it’s a combination of factors”. I wrote that prediction down before running the test. It came out exactly that way, which is the closest a researcher gets to smug.
So, why do chatbots say “not X but Y”?
First, some bad news: “not X but Y” is spreading.
I counted uses of “isn’t just” in English-language news since 2017, using GDELT, a project that reads the world’s news sites so the rest of us don’t have to. In the autumn of 2023 “isn’t just” started to climb, and by July 2025 it was turning up three and a half times as often as it had in 2019 to 2022.
Since that peak “isn’t just” has fallen by more than half, possibly because it got recognizable and irritating enough to edit out.
By 2026 “isn’t just” had reached the official websites of some of the largest companies on Earth. Word for word:
Google: “Government AI adoption isn’t just a technical upgrade, it’s a way to solve real world challenges…”
Salesforce: “…giving back isn’t just something we do — it’s who we are.”
NVIDIA: “The bottleneck isn’t just data — it’s the entire data factory.”
McKinsey: “AI investment isn’t just fueling innovation; it’s reshaping competitive dynamics…”
Amazon, quoting the head of MGM+: “What makes this bundle so compelling isn’t just the incredible value — it’s the curation.”
Spotify, from its co-CEOs: “AI isn’t about automation or taking your hands off the wheel; it’s about agency.”
None of this says who or what wrote them.
But who taught the LLM to use “not X but Y”?
Short answer: incentives, which is the short answer to most things.
Here is how a chatbot gets made, roughly, and where the “not X but Y” habit gets reinforced at each stage.
First, the LLM reads us. A chatbot starts life reading an enormous amount of human writing, and human writing, I regret to inform you, has been doing this for a century: “is not just” got about fifteen times as common in printed English between 1920 and 2019.
Then, the LLM gets asked about meaning. Ask Gemma why a village duck pond matters and 3 answers in 10 use “not X but Y”. But ask it to describe a watermill grinding flour and “not X but Y” is only 1 in 200. Questions about meaning demand insight, and “not X but Y” is the cheapest insight money can buy.
Finally, the LLM gets trained to be helpful. Gemma’s training recipe is private, so I borrowed a model whose every stage is public: OLMo 2, from the Allen Institute for AI. I asked each stage above in the training arc the same 152 questions.
Teaching the raw model to follow instructions made five times as many answers use “not X but Y”, which sounds damning until you notice the answers also got six times longer. Word for word, use of “not X but Y” barely changed; the habit just had more room to stretch its legs.
The next stages are where it gets interesting. The Allen Institute for AI then trains the model on preferences: show the LLM two answers, reward the better one, repeat many times.
After this training round, 1.7 times as many of its answers to questions about meaning used “not X but Y”. Questions about how things work stayed flat. Something in the grading of LLM answers seemed to like this.
So I went to find the grader.
People. On LMArena, a site where people vote for the better of two chatbot answers, I found nearly 5,000 head-to-heads where only one answer used “not X but Y”. “Not X but Y” won 56% of the time. Suspicious! Except it was usually the longer answer, and people like longer answers. At the same length, it was a coin flip.
An AI judge. This is the automated judge that grades answers during training, because no team of humans could read that many. It has the patience of a saint and the taste of a motivational poster. I took 600 real chatbot answers and rewrote one sentence in each so it said the same thing without the swerve. “Coffee culture isn’t just about drinks; it’s about community” became “Coffee culture is about drinks. It’s also about community.” An open reward model preferred the original three times in four. Against the bare “Coffee culture is about community”, it shrugged.
The AI judge’s weakness is oddly specific: it marks you down for admitting the boring part. “Not X but Y” lets you mention the boring part and wave it away in one breath, which makes it a safer sentence.
So: we wrote it for a century, training gave “not X but Y” room, and an AI judge likely helped to reinforce the construction. You get what you reward, and we rewarded this.
What you can do now you know this
If you write with AI, ask for the how and the what: steps, scenes, facts. Questions about why something matters are an open invitation to “not X but Y”. Showing the model a couple of examples in the style you want works better than asking it to avoid a phrase; banning a phrase is like banning a weed by name. And when you edit, look for the shape, since deleting one phrase lets another pathway pull it back to life.
If you build with AI, a writing tic like this can be ablated inside the model, cheaply, if you look in the right place. Then check what else you ablated. And if you train a reward model, maybe let it hear the boring sentence out.
Try it
The interactive version is at notthisbutthat.hopski.co: play an answer word by word with the 25 features on and ablated, and watch them light up on the map. The map lives in Neo4j, where each feature is a node linked to the features it fires alongside. Code, data and every result, including the tests that failed, are at github.com/ho3h/not-this-but-that.
Notes
Setup. gemma-2-2b-it with Gemma Scope layer_20/width_16k/canonical (Lieberum et al., 2024); gemma-3-4b-it with Gemma Scope 2 (gemma-scope-2-4b-it-res, layer 22, 16k). Chat template throughout. Ablating a feature subtracts only its contribution at that layer and leaves the rest of the model untouched; amplifying multiplies that contribution; clamping replaces it with a fixed value. Each ablated answer shares its random numbers with its normal twin.
Numbers. “Not X but Y” means the negated family (“not just X, it’s Y”, “isn’t X. It’s Y.”, “not only X but also Y”) or “more than just X. It’s Y”, counted with the project’s detector (v3). Ablating: 88 → 3 of 510 answers (17% → 0.6%, −97%) and 94 → 7 (−93%), pre-registered. Leaderboard: 40 “isn’t just” moments, top 25 of the ranking. Gemma 3: 156 → 27 (−83%). Training: 52 vs 11 of 250 answers. Question type: 234/800 vs 2/400 (Gemma 2), 294/400 vs 26/200 (Gemma 3). News: GDELT, English-language sources, “isn’t just” per 1,000 articles containing “according to”, 12.0 (2019–22) to 30.8 (2025–26), peak July 2025. Books: Google Books Ngram, “is not just” 0.32 (1920) to 5.08 (2019) per million words. Training stages: OLMo 2 1B base, SFT, DPO and Instruct, 152 questions × 4; meaning questions 9.7% (SFT) to 16.7% (Instruct), 1.72× (95% CI 1.09–2.92). Judges: 4,917 one-sided LMArena battles, 55.9% raw, odds ratio 1.01 adjusted for length; UltraFeedback and OLMo 2 preference data, 51% at equal length; Skywork-Reward-V2-Qwen3-1.7B on 600 rewritten answers, 76.7% vs the same idea without the contrast, 53.2% vs the bare claim. Every number has a receipt in the repo’s reports folder.
Sources. Templeton et al., “Scaling Monosemanticity” (Anthropic, 2024) and “Golden Gate Claude” (Anthropic, 23 May 2024). McGrath et al., “The Hydra Effect” (2023). Paech et al., “Antislop” (ICLR 2026). Kuznetsov et al., “Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders” (2025). Robert Lowth, De sacra poesi Hebraeorum (1753). Team OLMo, “2 OLMo 2 Furious” (Allen Institute for AI, 2024). Chiang et al., “Chatbot Arena” (2024). Cui et al., “UltraFeedback” (2023). Skywork-Reward-V2 (Skywork AI, 2025). Michel et al., “Quantitative analysis of culture using millions of digitized books” (Science, 2011). The GDELT Project. Juzek and Ward, “Why Does ChatGPT ‘Delve’ So Much?” (COLING 2025).