AI Research
We're an AI research and implementation lab. We test everything on ourselves before it reaches a client, then publish the numbers. We keep coming back to how little frontier model a job needs, where long context breaks, whether an AI judge can be trusted, and how much better AI gets when it knows your business.
Most critical findings
All notes and reports- 2xOwner rules written into a spreadsheet roughly doubled AI answer scores.
- 2.5%A cheaper model directed by a stronger one hit 98% of the quality on 2.5% of the frontier tokens.
- 94.4%Recall on Claude Sonnet 5 at 730,000 tokens, down from 100% at 34,000.
- 2.5xTwo AI judges read the same document from the same brief. One raised 15 numeric claims, the other 6.
- 23%Fewer context tokens after an agent ran 60 retrieval experiments overnight, with zero lost answers on the hidden test set.
- $0.14Per coding task for Kimi K3, against $0.44 for Claude Fable 5 and $1.62 for GPT-5.6-Sol.
All Notes & Reports
Owner Rules Written Into a Spreadsheet Roughly Doubled AI Answer Scores on 12 Synthetic Businesses
We tested Sheet Geek, our open-source spreadsheet skill, on 12 new synthetic businesses in a pre-registered, blind-judged study. Scores rose from 5.25 to 10.32 out of 15, and we list every limit we found.
Read more →98% of the Quality for 2.5% of the Frontier Tokens
Across 81 runs, a weaker model directed by a stronger one scored 96.8 against 98.8 for the stronger model alone, on 2.5% of its tokens. With a codebase map added, its best run tied.
Read more →Over-Engineering
We audited our own agent harness for three days and filed 120 issues, all of them true. They kept arriving for as long as we kept agents looking, so the fix was a budget.
Read more →AI Benchmarks
Two 2026 audits found broken answer keys in public AI benchmarks and a grader that passed 62.81% of wrong answers. The useful question about any score is whether you can run the test yourself.
Read more →Claude Code Context Window
A fresh Claude Code session spends 20,000 to 30,000 tokens before your first word. What loads, what range is healthy, and why to measure at the wire before cutting anything.
Read more →Context Rot: We Measured Recall at 700,000 Tokens
In 128 trials on Claude Sonnet 5, recall fell from 100 percent at 34,000 input tokens to 94.4 percent at 730,000, on a task whose difficulty never changed.
Read more →We Open Sourced the Harness. One Cost Question, Four Answers.
We made model-eval, our coding-model harness, public. Over 267 runs, one cost question asked four defensible ways gave four different answers, and we retracted a headline we had published.
Read more →Two AI Judges, One Document, Two Different Reviews
Two AI judges reviewed one document from an identical brief and made 15 numeric claims against 6, and 31 bullet points against 88. Review only stopped after we wrote four rules for what done looks like.
Read more →We Rebuilt 32 Pages Around One Question: Will an AI Cite This?
Generative engine optimization means structuring content so AI answer engines can quote it. This is the page structure we rebuilt 32 articles around, and how we measure it.
Read more →We Ran Four Agentic Coding Tools Through the Same Gate
We run Claude Code, Codex, Cursor and a local open-weight model on real client work. Whether code ships depends on the harness around each one more than on its model.
Read more →We Let an Agent Run 60 Experiments on Our RAG Stack While We Slept
An agent ran 60 controlled experiments on our hybrid retriever overnight while a supervisor checked each one. Context tokens fell 23% with accuracy inside tolerance and no lost answers on the hidden set, and the waste turned out to be padding inside the chunks rather than their ranking.
Read more →The Frontier You Can't See: Why "China Caught Up" Is Half a Myth
In summer 2026 Washington throttled America's best AI models and Anthropic's top tier went partner-only, while cheap Chinese open weights were said to have caught up. We separate what is measured from what is contested. Full paper attached.
Read more →Does an LLM Understand Credible Punishment?
We played a frontier model against a pricing rival that punishes any undercut forever and varied the chance the game continues. It started cooperating right at the 50 percent point theory predicts, but in short games it gave up nearly the whole prize.
Read more →Kimi K3 vs the Frontier: A Private Coding Eval
We ran Kimi K3 against Claude Fable 5 and GPT-5.6-Sol on our own nine coding tasks over 194 runs. All three passed at near-ceiling rates, and K3 was cheapest at $0.14 per task.
Read more →The Great Model Dump: Cheap Chinese Intelligence and Which Assumption Breaks First
Chinese open-weight models sit months behind the US frontier and cost 85 to 97 percent less to run. A sourced market thesis on what that does to AI margins, with the full technical report attached.
Read more →The Agents Were Innocent: Field Notes From a Red Wave
Wave 10 of our overnight agent pipeline came back all red, and none of the failures traced to the agents. A format gate broken for weeks started working and flagged 18 files of old debt, and 2 of 3,354 tests failed on fixture dates that had aged out.
Read more →We Ran 154 Controlled Trials of Claude vs GPT. Both Scored Perfect.
We ran Claude Fable 5 against GPT-5.6 on nine proven-solvable coding tasks in a blocked full-factorial design with two blind judges. Both passed everything, so the test was too easy to separate them.
Read more →Claude Fable 5 vs. GPT-5.6 Sol: A Practitioner's Field Report
Claude Fable 5 and GPT-5.6 Sol compared on price, context and positioning, plus what our blind stress test of a production harness with Sol found and did not find.
Read more →Why AI Agents Fail on Tasks They Can Actually Do
AI agents fail on real work more often from a missing harness than from a weak model. We lay out the five layers of a harness and the published data on the gap.
Read more →Why AI Agents Claim They're Done When They Aren't
AI agents call a task finished too early when nothing outside the model checks the work. These are the four fixes we use to stop it.
Read more →Running AI Coding on Matt Pocock's Method, in Production
Our AI coding pipeline follows Matt Pocock's method: interrogate the plan, lock a spec, cut small tickets, write tests first, and review on two axes before anything merges.
Read more →Token Efficiency: Spending an AI Context Budget Wisely
Token efficiency means treating context as a budget. Progressive disclosure, subagent delegation and prompt caching lower the cost of each task, and caching alone discounts repeated tokens by up to 90 percent.
Read more →When to Fan Out Subagents: Orchestration Patterns That Work
When it pays to fan work out to subagents, how to bring verification back to one place, and what breaks when orchestration is careless.
Read more →How to Optimize Content for Google AI Mode Search
Google's own AI Mode usage report points to four content signals: question headings, content that completes the task, follow-up questions and multimodal content.
Read more →How to Write Instruction Files AI Agents Actually Follow
CLAUDE.md and AGENTS.md files tend to grow into encyclopedias the agent stops reading. A short router file with topic docs loaded on demand keeps them working.
Read more →Running AI Coding Agents Unattended, Without Waking Up to a Mess
Overnight AI agent loops fail in predictable ways. We run backlogs in waves with machine-checkable done gates, a budget cap, and human approval on anything that leaves the sandbox.
Read more →The Repo Is the Only Thing an AI Agent Actually Knows
An AI agent knows only what a fresh session can read in the repo. The fresh-session test, and how to write state files an agent can trust.
Read more →Do AI Detectors Actually Work? What the Data Shows
OpenAI shut down its own AI detector in 2023 for low accuracy. We go through the published accuracy and false-positive data on the rest, and why a detector score should be treated as a hint.
Read more →Will AI Take Over the World? An Honest Answer
What AI researchers, superforecasters and the public say about existential AI risk, with sources. Researchers put a median 5% on an extinction-level outcome; top forecasters put it under half a percent.
Read more →Will AI Take Your Job? What the Data Says by Profession
BLS projections and research data on AI and jobs for lawyers, accountants, doctors, teachers and radiologists. The risk concentrates in entry-level, high-repetition work.
Read more →How Much Water Does AI Actually Use?
Published water-per-query estimates range from 0.26 mL to 500 mL. We trace where that gap of nearly 2,000x comes from.
Read more →What Is Artificial Intelligence, in Plain English?
A plain-English definition of AI, how it works, and how AI, machine learning and generative AI differ.
Read more →Who Invented Artificial Intelligence? The Real History
No single person invented AI. A sourced timeline from Alan Turing's 1950 question to the 1956 Dartmouth workshop and on to ChatGPT.
Read more →Is AI a Bubble? The Numbers Behind the Question
Capex, revenue and valuation numbers from 2025 and 2026, with the bull and bear cases side by side. A real correction already hit in June 2026.
Read more →The Best Free AI Tools in 2026 (Actually Free)
What ChatGPT, Claude and Gemini give you for free as of July 2026, and where each free tier stops.
Read more →Context Engineering
Context engineering is designing everything a model sees when it acts, including but not limited to the prompt. In our client work it has been the highest-leverage layer for reliability.
Read more →Harness Engineering
When an AI agent fails, we check the five layers around the model (task spec, context, environment, verification and state) before reaching for a bigger model.
Read more →AI Agent Evaluation
Benchmark leaderboards can shortlist models but cannot tell you whether your agent works. We test the whole system against real tasks, the way you would test software.
Read more →React Three Fiber in Production
What React Three Fiber is, when to pick it over vanilla Three.js, and what we learned running it after launch.
Read more →Three.js Examples for Business Sites
Ten categories of 3D work that serve a business website, each with a note on what it costs to run on a visitor's phone.
Read more →How Scroll Animation Websites Work
How scroll position becomes an animation timeline through video scrubbing, frame sequences or ScrollTrigger, and the performance rules that keep it smooth.
Read more →