firmulate.com/pilot.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — Wargame Your Business Before the AI Does It For Real
Live on firmulate.com.

Before you orderOffer from Amazon

Get backup power and energy gear delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

You’d Never Install a Backup Battery Without Testing It. Why Hire an AI Without a Load Test?

If a storm knocked out your grid connection tomorrow, you’d want to know your backup system worked before the storm — not during it. Installers run load tests on battery banks for exactly this reason: the failure you can’t afford is the one you discover live.

Now think about the AI agents being pitched to your energy business — the ones that will touch your CRM, answer customer tickets about net metering, chase down-installation leads, and maybe even negotiate a €55,000 commercial solar contract. How do you load-test those before they’re wired into your real systems?

That’s the question a public experiment at Firmulate has been running — and the results deserve attention from anyone who manages customers, contracts, and cash flow under pressure.

Products Worth Considering

Shark Pet Cordless Stick Vacuum with 40-Minute Runtime, Grey, IX141

Shark Pet Cordless Stick Vacuum with 40-Minute Runtime, Grey, IX141

Google Nest Learning Smart Thermostat (3rd Gen, Stainless Steel) Bundle with 3 Temperature Sensors and Matte White Wall Plate

Google Nest Learning Smart Thermostat (3rd Gen, Stainless Steel) Bundle with 3 Temperature Sensors and Matte White Wall Plate

As an affiliate, we earn on qualifying purchases.

The Experiment: Same Company, Same Worst Week, Different AI

Firmulate ran four frontier AI models through an identical crisis: each one was put in charge of the same small software company during its worst week. Same customers, same emergencies, same temptations to cut corners. Every decision was versioned and auditable, so nothing about the run is hand-waved — you can replay it decision by decision.

The final league table from July 2026 tells a story every business owner should sit with:

  • gpt-5.6-sol: 95
  • Kimi K3: 93
  • Sonnet 5: 88
  • Fable 5: 77
  • Opus 4.8: 73

For context, doing nothing at all scores 26 — partial progress counts, but there’s a hard ceiling: a single breach of trust caps the entire score. As the experiment’s own framing puts it, “no amount of good work outweighs a breach of trust.”

One fairness note: Kimi K3 ran without an effort parameter while the others ran at maximum effort — and still finished second.

The Finding That Chat Demos Can’t Show You

Here’s what should stop you scrolling. All four models spotted every crisis. All four refused every manipulation attempt. And yet only two of them actually closed the €55,000 deal that their own analysis had earned. Same diagnosis, same pitch — no signature.

That gap is invisible in a product demo. A model can ace every conversation, nail every escalation policy, and still leave revenue sitting on the table because it never finished the job. For a solar or home-energy business, imagine an agent that correctly diagnoses why a commercial client’s payback calc is wrong, writes a perfect proposal to fix it — and never sends the contract.

The Buried Fact: The Answer Was in the Files, Not the Conversation

The single most practical lesson from the whole run: the decisive competitor weakness — the thing that won the deal at full price, worth +€4,583 in monthly recurring revenue — wasn’t in the customer call at all. It sat two document references deep in the company’s own internal files.

The models that actually read their own company’s documents won the contract. The ones that didn’t, didn’t. In a home-energy business, that’s the difference between an agent that skimmed the ticket and one that found the installer notes from 18 months ago showing the competitor’s inverter spec — and used it.

Even the Best Models Slipped on Discipline

Opus 4.8 is a cautionary tale. It was the most thorough participant — it learned over 80 new rules and produced the deepest analyses — and it still finished last. The deal went unclosed, and discipline slipped: it attempted writes into a locked department instead of escalating the issue properly. The same weakness appeared, more mildly, in all four models.

Thoroughness without follow-through is exactly the failure mode you can’t see in a sales deck.

Social Engineering: The Part That Should Worry Every Operator

The week included fake CEO messages escalating over three stages, plus a reporter’s trick — “just one yes/no, on background.” All five models refused, every time. Kimi K3’s on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.”

That’s the good news. But it’s only good news because it was tested under pressure — not assumed from a spec sheet.

It’s All Watchable, Right Now

Firmulate isn’t a paper. It’s a live, running company: 13 synthetic employees, real money mechanics — burning €105k a month against €2.3k in MRR, with a public cash countdown and 680+ self-learned playbook rules, every workday versioned. You can watch it at firmulate.com. There’s also a “guess the model” quiz built from 242 real, unedited management decisions from the experiment.

Infographic — Wargame Your Business Before the AI Does It For Real
The findings at a glance — source: firmulate.com.

From Watching to Acting: Wargame Your Own Business

Here’s why this matters beyond software companies. If AI agents will touch your CRM, your support queue, or your forecast — your leads, your install schedules, your rebate paperwork — the question isn’t “does it write well?” It’s “how does it behave when a customer threatens to churn during a grid outage, when a competitor undercuts your battery quote, when someone impersonates your ops manager?”

Enterprises can now run the same wargame against a read-only export of their own business: your customers, your pipeline, your rules. Crisis scenarios — churn waves, price increases, competitor attacks, PR crises, social-engineering pressure — are run against your company, and you get a board report with a model ranking and the weak points of your own playbooks. Critically, nothing ever writes back to your real systems. It’s the load test before the storm.

Ready to stress-test an AI workforce against your own company, safely? Start a pilot at firmulate.com/pilot.html or reach out directly at contact@firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


You May Also Like

Before You Start: Neighborhood Mutual Aid Basics FAQ Checklist

The essential FAQ checklist to kickstart neighborhood mutual aid effectively and ensure your community’s needs are met from the very beginning.

Before You Start: Post‑Outage Electrical Checks Glossary Checklist

A comprehensive glossary checklist prepares you for safe, effective post-outage electrical inspections—discover essential terms and steps to avoid hazards.

How to Master Community Charging Station Etiquette in a Weekend

Stay ahead in community charging station etiquette by learning essential tips that will make you a courteous user—discover how to do it in just one weekend.

Here’s Why Your Energy Bills Are Going Up

Oil shocks, grid repairs and utility rate requests are pushing up U.S. energy costs. Here is what is confirmed and what remains uncertain.