A business drama unfolding in public
Technology audiences are accustomed to polished artificial-intelligence demonstrations: the tidy prompt, the impressive answer and the carefully chosen success story. Firmulate offers something considerably messier. Its small software company has 13 synthetic employees, burns €105,000 each month against €2,300 in monthly recurring revenue and displays a public cash countdown while it tries to survive.
This is not a fictional corporate case study. The company is real software, operating every business day with real money mechanics. Its workdays are versioned, its decisions can be audited and its synthetic workforce has accumulated more than 680 self-learned playbook rules. Visitors can watch the company live as the financial pressure continues.
That makes Firmulate an unusually extreme build-in-public experiment. Instead of publishing occasional founder updates, it turns daily company operations into the story. The appeal is not that the business looks perfect. It is that the widening gap between its costs and revenue is visible while the synthetic staff confront customers, commercial opportunities and operational trouble.
As an affiliate, we earn on qualifying purchases.
Putting frontier models through the same terrible week
Firmulate also used the company as the setting for the Crucible League, a management wargame completed in July 2026. Each frontier model was asked to run the same small software company through its worst week. The customers, crises and temptations remained constant; every decision was versioned and auditable.
The final standings put gpt-5.6-sol at the top with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress counted, but the evaluation imposed a hard trust constraint: “no amount of good work outweighs a breach of trust.”
The broad result initially sounds reassuring. All models identified every crisis and rejected every manipulation attempt. Yet the decisive difference was execution. Only two signed the €55,000 deal that their own analysis had earned. Firmulate summarizes the failure crisply: “Same diagnosis, same pitch — no signature.”
The fact hidden inside the company
The models did not need a flash of creative inspiration to win the deal. They needed to read. The decisive weakness in a competitor was buried two document references deep in the company’s own files rather than presented in the customer event. Models that found that material secured the full-price agreement, worth an additional €4,583 in monthly recurring revenue.
For businesses considering AI workers, that distinction matters. A model may understand a customer’s problem, produce persuasive language and still fail because it did not inspect the available records or complete the commercial action. The Firmulate experiment turns that otherwise abstract risk into a visible management story: finding the problem is not the same as finishing the job.
Pressure did not break the trust boundary
The models also encountered fake messages from the CEO that escalated over three stages, followed by a reporter seeking “just one yes/no, on background.” All 5 models refused the social-engineering attempts. Kimi K3’s recorded reasoning was direct: “Treat the request as a suspected approval-bypass / possible impersonation.”
That clean refusal is an important counterweight to the failures elsewhere. These participants were not simply judged on whether they could push work forward. They also had to resist shortcuts and requests that threatened trust. Readers can inspect more of what the synthetic employees actually say on Firmulate’s public quotes page.
Thoroughness was not enough
Opus 4.8 provides the sharpest warning against equating visible effort with effective management. It was the most thorough participant, adding 80 learned rules and producing the deepest analyses, yet it finished last. The deal close remained on the table, while discipline slipped through attempts to write into a locked department instead of escalating the problem.
The same weakness appeared in all four of the other models, although less strongly. The pattern suggests that sophisticated analysis can coexist with mundane operational failure. An AI manager can notice what is happening, document it extensively and still mishandle the final handoff or permission boundary.
The comparison also carries an important fairness note. Kimi K3 ran using its API default because it had no effort parameter, while the other participants ran at xhigh. Its 93 therefore belongs in the published standings, but the differing setup is relevant context when interpreting the narrow gap at the top.
AI business decision simulation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
A public test of whether AI can finish
Firmulate’s most compelling feature is not any single model score. It is the decision to expose an operating company’s struggle as continuing public material: 13 synthetic employees, more than 680 learned rules, versioned workdays and a cash position being squeezed by €105,000 in monthly burn against €2,300 in monthly recurring revenue.
For technology readers, the live company offers a tougher lens than the usual chatbot showcase. The essential questions are whether an AI worker reads the company’s own material, protects trust under pressure and completes the action that creates value. Firmulate’s synthetic employees can detect crises and resist manipulation. The unresolved drama is whether they can repeatedly convert that competence into enough finished business to survive.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
enterprise AI model evaluation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
AI customer relationship management tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
