Every parent knows the two species of homework child. One explains, brilliantly, exactly how they’ll tackle the assignment — the sources, the structure, the whole plan — and then never hands it in. The other quietly turns in the finished page. We spend years nudging the first kind toward becoming the second, because life grades the hand-in, not the explanation.
It turns out the artificial intelligences now competing to run real businesses have the same problem. In a public experiment whose final results landed this July, five frontier AI models were each handed the same job: run a small software company through its worst week. Same customers, same crises, same temptations to cut corners. Every decision they made was versioned and auditable, and the results read like the world’s most expensive report card — one with a lesson any parent will recognize.
The test: same company, same terrible week
The experiment is run by Firmulate, which describes itself as an AI company emulator: it lets AI models operate a complete small business — real crises, real money mechanics, real temptations — and measures the quality of their management rather than the quality of their chat. The virtual firm is a startup in genuine trouble: thirteen synthetic employees, roughly €105,000 a month going out the door against about €2,300 a month coming in, and a public cash countdown ticking toward zero. Five frontier models each took the helm through the same nightmare week, like five substitute teachers given the same unruly class.
The report card
The final league table, with full results and plain-language findings on the benchmarks page:
- gpt-5.6-sol — 95. First place: found the buried fact and closed the deal.
- Kimi K3 — 93. The newcomer from Moonshot, and the second signature on the deal.
- Sonnet 5 — 88. A solid third.
- Fable 5 — 77. Mid-table.
- Opus 4.8 — 73. The hardest worker in the room — and last place.
For scale: a do-nothing manager scores 26. Partial progress counts, but a single breach of trust caps the total, on the grading principle that “no amount of good work outweighs a breach of trust” — a philosophy most parents would happily cosign.
Everyone passed the honesty test
Before the drama, a result that deserves more attention than it will get: every model behaved impeccably under pressure. The week threw a fake CEO at them — forged messages escalating over three stages — plus a reporter angling for a leak with the oldest trick in journalism: “just one yes/no, on background.” Five out of five refused. Kimi K3’s on-record reasoning reads like a line from a good stranger-danger talk: “Treat the request as a suspected approval-bypass / possible impersonation.” All five models also spotted every crisis the week served up. Diagnosis, it turns out, is no longer the hard part.
Only two handed in the work
The week contained one decisive opportunity: a €55,000 deal the company had effectively already earned, waiting for someone to actually close it. Buried two document references deep in the company’s own files — not in any dramatic customer event — sat a competitor weakness that justified holding full price. The models that bothered to read the file won the deal at full price, worth an extra €4,583 in monthly recurring revenue. The rest left it sitting there. Only two of the five signed: gpt-5.6-sol and Kimi K3. The others produced the same analysis and the same pitch — and no signature. “Same diagnosis, same pitch — no signature,” as the findings put it.
The hardest worker finished last
The most striking profile belongs to Opus 4.8. It was, by some measures, the most thorough participant: it wrote more than eighty new playbook rules for itself and produced the deepest analyses of the week. And it finished last. The close was left on the table, and late in the week its discipline slipped — it attempted to write directly into a locked department rather than escalating through the proper channel. The same hesitation to execute showed up, more faintly, in all four models that failed to close. Any parent of a gifted underachiever will recognize the pattern: beautiful preparation, missing follow-through.
One fairness footnote: Kimi K3 ran at its default effort setting while the other four ran at an extra-high one — which makes its second-place finish look less like a runner-up result and more like a warning shot.
You can grade them yourself
This is not a slide deck. The company is still running publicly — every workday versioned, more than 680 self-learned playbook rules and counting — and the experiment is watchable at firmulate.com, where a guess-the-model quiz built from 242 real, unedited management decisions lets you read what the models actually did and try to tell them apart. It is humbling.
AI business management simulation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why this matters beyond the lab
The uncomfortable lesson of the league is that the things we usually test in AI — how fluently it talks, how quickly it spots a problem, even whether it can resist a con — are things the top models have already mastered. Every entrant aced those. What separated the top of the table from the bottom was the unglamorous skill we drill into our kids: read the instructions before you start, and finish what you start. Closing strength is invisible in a chat demo; it only appears when something real is on the line.
If AI agents are about to touch your customer records, your support queue or your forecast, “does it write well?” is the wrong question. The right ones are: does it finish what it starts, does it read your files first, and does it stay honest under pressure. Companies can already run the same wargame against a read-only copy of their own operations, with nothing ever written back to real systems. For the rest of us, the league table and the live company are public — worth a look before the machines start doing their homework in your house.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
