Bullshit Detector
Agent skills that fact-check the internet.
Point your agent at a viral video, article, tweet or PDF. Get every claim back, checked, scored, and sourced.
- Confirmed
- Plausible
- Misleading
- False
- Unverifiable
No source, no verdict. Not even when the model is sure.
Install
Portable Agent Skills are plain markdown plus self-contained Python. Works in Claude Code, Codex, GitHub Copilot, Cursor, Gemini CLI, OpenCode, Zed, and any harness with web search. No API keys. MIT licensed.
Any agent
npx skills@latest add SerhiiKorniienko/bullshit-detector
Claude Code plugin
/plugin marketplace add SerhiiKorniienko/bullshit-detector
/plugin install bullshit-detector@serhii-korniienko
No terminal — Claude Cowork
In the Claude app: Customize → Plugins → Add marketplace, paste
SerhiiKorniienko/bullshit-detector, install from
the list. Everything happens by pasting a link.
Pick one, not both. Then ask your agent: “is this bullshit? <url>”
Per-app walkthroughs and a what-works-where matrix live in the setup guide.
What a report looks like
“Researchers built an AI that has to earn its own salary or go bankrupt”
Tweet · 14 claims extracted, 14 individually source-checked · run 7m2s
| Claim | Verdict | Evidence |
|---|---|---|
| Payment = quality × estimated hours × actual BLS wage | ✅ confirmed | README, verbatim. This is the tell the tweet buries: the income side is simulated dollars from US Bureau of Labor Statistics wages, no client pays anything. Only the token costs are real money. |
| Agents earned $10,000 in 7 hours | 🟡 plausible | [2 URLs → 1 origin] Matches the lab’s own launch headline from six months earlier, undated in the tweet. Self-reported; no independent replication found. |
| “This isn’t a benchmark, it’s a survival test” | 🟠 misleading | The project’s own materials call it an economic survival benchmark on OpenAI’s GDPval dataset. The sentence does the work of hiding that the salary is benchmark-denominated. |
Every row links its source. A script recounts the finished report’s own arithmetic and rejects it if the tally doesn’t reconcile.
What it doesn’t do
Honest limits, because a tool like this earns nothing by overselling itself.
- → It checks premises, not reasoning. A false fact gets caught; a bad inference drawn from true facts sails straight through.
- → It can only cite what it can reach. Many high-reputation outlets block agent crawlers, and SEO content ranks in the gap. The measurements are published; they are worse than I assumed.
- → It has no eval harness yet. So there is no number for how often it’s right. Someone on Hacker News asked for the obvious test, it ran on its own README and scored 3/10, catching a stale API price.
How it works
1 · Fetch
One script turns any URL into clean text plus metadata: YouTube transcripts, TikTok captions, articles, PDFs, tweets. No API keys, no paid endpoints.
2 · Extract
Every discrete claim is pulled out and split into load-bearing, meaning the thesis dies without it, and incidental. Ambiguous claims are dropped, and the count of drops is published.
3 · Verify
Each claim gets its own web search. Verdicts require a source; the skill is forbidden from confirming or refuting from model memory. Reprints of one wire story collapse to one origin.
4 · Score
A 0–10 BS score, a hype-signal scan, and an incentive analysis: who benefits if you believe this. Plus the numbers a hostile reader would check first.
Seven skills, one repo
bullshit-detector
Claim-by-claim verdicts, hype scan, incentive analysis, 0–10 score.
fetch-content
Any URL to clean text and metadata. YouTube, TikTok, articles, PDFs, tweets.
coverage-check
Counts the independent origins behind “widely reported”, not the URLs.
summarize
TLDR, key points with timestamps, notable quotes, who it is for.
explain
Any concept in the content, at the depth you ask for, with a glossary.
report-card
Renders a finished report as a shareable page that prints cleanly.
share
Turns a report into posts and a branded image carousel.
Try it on something viral
Free, MIT licensed, no API keys, no account. It runs on your draft too. Point it at a post you’re about to publish and it flags what a hostile reader hits first.