Automate up to the decision, and never the decision itself

X’s open-source ranking model exposes nineteen prediction heads. None of them is an impression. Four of them are negative feedback, and the live Rust scorer does something with those four that is stranger than the folklore. A post whose weighted score comes out net-negative is not ranked low on the same scale as everything else. It leaves through a different branch, gets rescaled by the ratio of the negative weights to the total, and is then multiplied by a constant called NEGATIVE_SCORES_OFFSET that the repository imports and never defines. If that constant is positive, every net-negative post lands in a band underneath every net-positive one.

// home-mixer/scorers/ranking_scorer.rs:83
let negative_sum = -(not_interested + block_author + mute_author + report + not_dwelled);
let total_sum = positive_sum + negative_sum;

// home-mixer/scorers/ranking_scorer.rs:175
fn offset_score(combined_score: f64, w: &ScoringWeights) -> f64 {
    if w.total_sum == 0.0 {
        combined_score.max(0.0)
    } else if combined_score < 0.0 {
        (combined_score + w.negative_sum) / w.total_sum * NEGATIVE_SCORES_OFFSET
    } else {
        combined_score + NEGATIVE_SCORES_OFFSET
    }
}

I opened that file to work out whether an automated welcome DM was worth its click rate. It was not. The click rate turned out to be the least interesting reason, and the most interesting one had been written down years before I ran it.

What I was running, and the line that started this

I run cold sequences for 1688 sourcing at Yakkyo S.p.A. One of them got a one-line reply: “Is this a real person or a bot?”

The email was technically perfect. Right name, right company, a reference to their product category that Clay had pulled and Claude had stitched into a clean opening. It read like nobody was home. Reply rate had been sliding for six weeks and I had been blaming the copy.

I take that question seriously rather than laughing it off, for a specific reason. From 31 October 2025 to 27 April 2026 I ran a publishing bot against this domain. It made 842 Add post: commits and produced 2,922 machine-generated files before I deleted them; of the 739 English ones, 180 cited a source called “Internal Analysis” that does not exist. The full account is here, and those counts come from a private repository, so you have my word rather than a link.

Same shape, different scale: an automation took over a judgment call, kept producing output that looked fine, and nothing alerted because nothing broke. So this quarter I killed automations instead of stacking them. Three went, eleven stayed. Every figure below for my own results is my own unaudited number from my own work, published first on X.

Kill 1: the fully automated cold sequence

Clay enriched the lead, an n8n workflow scored it, Claude wrote a three-email sequence with a first line pulled from the prospect’s site, and it sent on a schedule. I forgot about it for about two months.

Reply rate fell from roughly 9 percent to under 4. What diagnosed it was not the average: positive replies fell faster than total replies, while “unsubscribe” and “not relevant” kept arriving at the same clip. The sequence was filtering out the people most worth talking to and retaining the ones who were going to say no anyway.

Why it happened. Automating a first line does not scale the good version of that line. It scales the mean of every line the model would write, and that mean is competent and interchangeable. Cold outreach does not pay on the mean. It pays on the tail, where one specific observation lands because somebody looked. Taking the human out of that step does not lower quality evenly. It removes variance, and the variance was the product.

How to detect it. Reply rate sums over both tails, so the drift hides inside it. Split it. Positive and negative replies are separate series that move independently, and their divergence leads the average by weeks.

Then watch what your provider watches. Google’s sender guidelines are specific and almost everyone quotes them wrong. The number people repeat is 0.3 percent; the guidance says “Keep spam rates reported in Postmaster Tools below 0.10% and avoid ever reaching a spam rate of 0.30% or higher.” Two thresholds, two jobs. 0.10 percent is the target you sit under, 0.30 percent is the line you never touch, and treating 0.30 as your budget spends your margin before the bad week arrives.

Making that measurable needs a clean opt-out channel: RFC 8058 one-click unsubscribe, the List-Unsubscribe-Post: List-Unsubscribe=One-Click header Google names directly. Without it, an annoyed recipient’s cheapest exit is the spam button, which routes the signal into your reputation instead of your dashboard. With it, opt-outs are a dated count you can line up against the day an automation went live. For US recipients that channel is also statutory: 15 U.S.C. 7704(a)(4)(A)(i) makes it unlawful to send more than 10 business days after an opt-out request.

What runs now. Enrichment and scoring stayed automated. That genuinely took two hours and now takes about eleven minutes. I am leaving that unrounded and unconverted, because a percentage computed off an “about” would carry a decimal place I never measured. Claude drafts the full email in maybe 20 minutes and I rewrite the opening of every tier-one email by hand. Reply rate came back to around 8 percent, and the positive-reply mix is better than it ever was on full auto. The judgment moved back to a human. The typing stayed automated.

A gate that did not exist when I built this. Article 50 of the EU AI Act, Regulation (EU) 2024/1689, entered into force on 2 August 2026, and I work in Messina, so it reaches me. It is also being widely misreported as a blanket duty to label AI-written email. Article 50(1) puts the interaction-disclosure duty on providers of systems intended to interact directly with natural persons. Article 50(4)’s deployer duty on text covers material “published with the purpose of informing the public on matters of public interest” and carves out anything that “has undergone a process of human review or editorial control”. A cold email drafted by a model and rewritten by me fits neither well. Read it before you panic or dismiss it: “real person or a bot?” now has a legal surface beside the craft one.

Kill 2: the automated DM to every new follower

Every new follower on X got an automated welcome DM with a link. A webhook fired the message. It is the growth tactic everyone copies.

The DMs converted to clicks at well under one percent, and mute and block events on the account ticked up in the exact window I turned it on. So I killed it on the numbers. Then I read the policy and found the numbers had been the second-best reason all along.

X’s Developer Agreement and Policy, under “Spam, bots, and automation”, says that “Services that perform write actions, including posting Posts, following accounts, or sending Direct Messages, must follow the Automation Rules. In particular, you should:”. The first bullet under that colon is:

Always get explicit consent before sending people automated replies or Direct Messages

Read the modal verbs, because they are doing two different jobs. The must points at a separate document: that bolded sentence is a link to help.x.com/rules-and-policies/x-automation, which answers 403, with a Chrome user agent as readily as without one, so I have not read the Automation Rules and will not quote them. The consent line is a should, and it is X writing out in its own words what it expects compliance to look like. A follow is not explicit consent on either reading. Nobody who clicked that button agreed to receive anything. If the automation authenticated through the X API, it ran its whole life against the one instruction X spelled out on a page I can fetch, independent of how it performed. The same section adds a line that reads differently after the cold-email story above: “You should never mislead or confuse people about whether your account is or is not a bot.”

One boundary. That section governs write actions through the X API and developer products. An automation driving a browser falls under the X Rules on platform manipulation instead, which is help.x.com/rules-and-policies/platform-manipulation and also 403s, so the same non-quotation applies there.

The mechanism, folklore removed. Growth posts repeat a table of weights: reply-back plus 75, mute or block minus 74, report minus 369. Those constants appear nowhere in either public repository. What is published is a demo fusion in phoenix/run_pipeline.py at lines 355-360, containing four heads, all positive:

weighted = ( all_probs[:, IDX_FAV]   * 1.0
           + all_probs[:, IDX_REPLY] * 0.5
           + all_probs[:, IDX_RT]    * 0.3
           + all_probs[:, IDX_DWELL] * 0.2 )

Production weights resolve at request time from a feature-switch service that is not vendored. No negative weight has ever been published, so nobody, me included, can honestly say what the system punishes hardest. The same gap covers NEGATIVE_SCORES_OFFSET from the opening code fence: ranking_scorer.rs pulls it in with use crate::params::* and the published tree has no params file. I pulled the whole repository listing, 244 paths, to check that. The branch is verifiable and the magnitude is not, which is the honest state of every weight discussed in this section.

What is verifiable is better than a weight, because it is structural rather than tunable. phoenix/runners.py declares nineteen heads at lines 233-252 and, at lines 266-271, NEGATIVE_FEEDBACK_INDICES = [14, 15, 16, 17]: not_interested_score, block_author_score, mute_author_score, report_score. “Show less of this” is not_interested_score. Each is a separately modelled prediction, and offset_score sends any candidate that nets out below zero down the separate rescaled branch rather than the shifted one.

The account-level claim, corrected. I used to say negative feedback drags down how the whole account gets treated. The outcome is roughly right and the mechanism I described does not exist. There is no account penalty ledger in the open source. block_author_score and mute_author_score are per-candidate, per-viewer predictions that this viewer would block or mute you over this post. The genuine account-level effect is blunter and permanent: author_socialgraph_filter.rs removes your post from a viewer’s candidate set outright once that viewer has muted or blocked you, and it propagates through quotes and retweets. One mute and you are gone from that feed at any score, and no amount of later quality recovers it.

There is also an asymmetry worth knowing, one line in each of two files. Scoring history defaults to UserActionAggregationType::DenseWithNotInterestedIn (scoring_sequence_query_hydrator.rs:47). Retrieval history defaults to plain Dense (retrieval_sequence_query_hydrator.rs:50). Explicit negative feedback is modelled by default where posts are scored, and not by default where they are fetched.

And the part that kills the tactic on its own terms. Outbound DMs are not a ranking input at all. The only DM among the nineteen heads is share_via_dm_score at index 8, a positive that models somebody sharing your post via DM. Sending DMs earns nothing. The harm route was mute and block plus a policy breach; the upside route did not exist.

What runs now. Nothing automated. I reply to people’s posts, and when someone follows after a good exchange I send a real DM by hand, maybe one in ten. Slower, and those conversations go somewhere. X API access has since moved to pay-per-usage pricing, so the tier names and prices in every guide written about this tactic are now wrong as well. Read the policy first, the pricing page second.

Kill 3: the AI-generated weekly content batch

Every Monday a workflow generated five posts and a thread from that week’s themes and queued them. Five minutes of work for a week of content. Impressions held steady and replies cratered.

My explanation for that was wrong, and the correct one is more useful. I said the cheap signal on X is the impression and the expensive one is the reply. There is no impression head. The nineteen-entry ACTIONS list contains no impression at all; raw counts are telemetry, not model inputs. Nor can I claim a reply outranks a like, because in the only fusion X has published a reply carries 0.5 against a favorite at 1.0.

What I was describing is dwell, which is modelled harder than anything else in the list: dwell_score at index 10, a discrete dwell_time at index 18, and again in the continuous head block. Then the live Rust path adds a head that is not in the nineteen-head list at all, not_dwelled, sitting inside negative_sum next to block, mute and report. Being seen and scrolled past is directly and negatively scored. That is what generated content produces. It reads fine and it skims, and skimming is the failure the model is instrumented to catch.

Corroboration sits one level up, in the content screen rather than the ranker. grox/classifiers/content/banger_initial_screen.py runs a Grok vision model over posts and returns a result whose fields include quality_score and, at line 37, slop_score: int | None. Somebody at X thought machine-generated filler deserved its own integer.

What runs now. AI writes a first draft in 20 minutes, I add my voice and the part I am genuinely unsure about, and I post three times a week instead of five. Fewer posts, more replies. The publish test is whether I would be slightly nervous to post it. No flinch, no reach.

The statistic everyone cites about this, and what it says

A 2025 MIT report gets quoted constantly for the claim that 95 percent of AI pilots never reach measurable return. I have quoted it that way myself. Having now read all 26 pages, the popular version is overstated on four axes, and the correction is worth more than the statistic was.

Table: What the MIT NANDA report actually is, against how it is usually cited.

The 95 is one hundred minus five, taken off the branch of a two-branch funnel that excludes the tools most people actually use. Quoting it as a verdict on AI means quoting the number that specifically is not about ChatGPT or Copilot.

One more checkable fact I have not seen mentioned: the report is no longer served from MIT. The Internet Archive’s index for nanda.media.mit.edu/ai_report_2025.pdf records four 200s on 18 August 2025, at 11:55:20, 12:50:35, 12:54:28 and 14:57:14 UTC, then a 404 at 17:28:56 UTC the same day. Its URL now lands on a Media Lab group overview page. The most-quoted AI statistic of 2025 has not been hosted at its published address since that afternoon.

One thing to get right if you check that index yourself: the length column is not the size of the PDF. It reads 886,480 on the first capture, 867,304, 886,477 and 865,562 on the next three, all four with the same content digest, because it measures the archive’s own compressed record. I downloaded the archived file. It is 923,623 bytes, and it is the same file in every capture.

My reading of it is that most people automate the judgment rather than the grunt work. That is my opinion, not a finding. Its nearest actual finding is budget misallocation, and it contradicts itself inside one section: the takeaway line says 50 percent of GenAI budgets go to sales and marketing while the body of that same section says “approximately 70 percent”. Cite it if you like, but say which number you took and that the source gives two.

The rule, and the paper that named it in 2000

The rule I run now: automate up to the decision, and after the decision, never the decision itself. That is a rediscovery, and the original is more precise than my phrasing.

Parasuraman, Sheridan and Wickens (IEEE Transactions on Systems, Man, and Cybernetics Part A 30(3):286-297, May 2000) model automation as applying to four independent stages: information acquisition, information analysis, decision and action selection, and action implementation. Level is chosen per stage. Their ten-point scale applies to stage 3 specifically, running from a computer offering several options, through level 4 where it suggests one alternative and the human retains authority, through level 6 where the human gets a limited window to veto, up to level 10 where the computer decides and acts alone.

Every automation I killed sat between levels 7 and 10 on stage 3. Every replacement sits at level 4. Enrichment and scoring are stages 1 and 2, and I automated those harder rather than less. That is what “buy the boring stuff, build the edge” means once you say it precisely.

Running stage 3 too high has its own named failure. Endsley and Kiris (Human Factors 37(2):381-394, 1995) called it the out-of-the-loop performance problem: an operator supervising an automated decision loses the situational awareness needed to take over when it degrades. I lost six weeks to that. I was not monitoring the sequence. I was receiving its output.

The quarterly check, and what each trip means

Every automation goes through this once a quarter. Two trips and it is killed or pulled back to level 4.

1. A quality signal falling while a volume signal holds. Track positive and negative replies as separate series; their divergence leads the average.
2. A negative signal rising since launch. Unsubscribes, mutes, blocks, “not relevant”, Postmaster spam rate against 0.10 percent rather than 0.30. Implement RFC 8058 so opt-outs arrive as a dated count rather than as reputation damage.
3. It is making a call a human used to make. That is stage 3. Name the level it runs at; above 4 is the finding.
4. I would be embarrassed if the recipient knew. The bot test, now also a compliance question in the EU and a policy question under any developer terms you have accepted.
5. It could break for three days without alerting me. If you would learn from a metric rather than an alarm, you are out of the loop in the Endsley sense.
6. The time saved is smaller than the trust it costs. Two hours down to eleven minutes with no quality cost, keep forever. Five minutes saved for a point of reply rate, kill it.

What I have now that I did not have in April

Four things, and only one is time.

A vocabulary with citations behind it, so “automate the boring stuff” becomes “keep stage 3 at level 4 and push stages 1, 2 and 4 as high as they go”. A compliance gate that runs before the ROI gate, because kill 2 was on the wrong side of the policy before it was on the wrong side of the numbers, and I checked those in the wrong order. A detection method that watches divergence between two series rather than the level of one. And a reading of the ranker built from files rather than from other people’s posts, which is why the version above corrects three things I used to say.

Two numbers in my original post do not reconcile, and I would rather fix that here than let it sit. I killed three automations and kept eleven. I also said most funnels are over-automated by about a third. Those are different claims and neither confirms the other: three of fourteen is a count of things I run, and the one-third is a guess about other people’s funnels. Read the count as a count and the guess as a guess.

The hours I got back did not go into building more automations. They went into the manual first lines, the real DMs, and the posts I am slightly nervous about. Those are the parts that do not scale, which keeps turning out to be the same list as the parts that work.