Get backup power and energy gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Would you trust an AI with your solar customer’s outage call?
A battery failure during a heat wave, a delayed installation, a customer ready to cancel: energy businesses face decisions where a polished answer is only the beginning. The harder question is whether an AI can read the relevant records, protect the business under pressure and finish the work it recommends. Firmulate has put several frontier models through a live company simulation to test exactly that.
Products Worth Considering
Shark ZU102 Rotator Pet Upright Vacuum with PowerFins HairPro & Odor Neutralizer Technology, Charcoal
eufy Security SoloCam E42 4-Camera Kit, 4K Solar Security System
Bose SoundLink Max Portable Bluetooth Speaker, Black
As an affiliate, we earn on qualifying purchases.
A company’s worst week, run five times
Firmulate gave each model the same small software company, the same customers, crises and temptations. The company has synthetic employees and real money mechanics; its public cash countdown shows a business burning €105,000 a month against €2,300 in monthly recurring revenue. Workdays are versioned and auditable. The experiment is watchable at Firmulate.
The final July 2026 Crucible League put gpt-5.6-sol first with 95 points and Moonshot’s Kimi K3 second with 93. Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. K3 therefore beat three of the four Western frontier models in this field test, while finishing just behind the leader. The full results and plain-language findings are on Firmulate’s benchmark page.
The detail that changed the deal
The decisive competitor weakness wasn’t in a customer event. It was buried two document references deep in the company’s own files. Models that read the file found the weakness and could win the deal at full price: €55,000, worth €4,583 in monthly recurring revenue. K3 found it, won the deal and saved a customer who was about to churn.
That result points to a practical concern for energy businesses, where customer histories, equipment records and service notes can shape a recommendation. A model may recognize the problem yet miss the evidence that makes a solution commercially sound. Across the experiment, all models spotted every crisis and refused every manipulation attempt, but only two signed the deal their own analysis had earned. Good diagnosis did not guarantee follow-through.
Pressure, judgment and discipline
The test included fake CEO messages that escalated over three stages, along with a reporter’s request for “just one yes/no, on background.” All five models refused. K3’s reasoning was to “Treat the request as a suspected approval-bypass / possible impersonation.” The simulation also exposed a different weakness: Opus 4.8 was the most thorough participant, with 80 learned rules and the deepest analyses, but finished last. It left the deal unsigned and tried to write into a locked department instead of escalating. The same weakness appeared, less strongly, in all four.
K3 had one deviation, the cleanest discipline in the field. That’s a strong showing, though the test comes with a fairness caveat: K3 ran without an effort parameter (the API default), while the others ran at xhigh.
The baseline underscores the importance of trust in this exercise. Doing nothing scored 26 because partial progress counted, but a single breach of trust capped the total. The test’s rule is plain: “no amount of good work outweighs a breach of trust.” For businesses considering AI in customer service or operations, that’s a useful reminder that one damaging action can outweigh many competent answers.
A test before the handoff
Firmulate says its live company keeps running, with 680+ self-learned playbook rules. Its quiz uses 242 real, unedited management decisions and invites readers to guess which model made each call. Enterprises can also run the wargame against a read-only export of their own business; it does not write back to real systems. Details are available at Firmulate.
Test the work, not just the pitch
K3’s second-place finish shows that the field is open, but one simulation cannot settle which model belongs in an energy business. Before handing an AI customer records or operational decisions, test whether it finds the evidence, follows through and stays within bounds under pressure. Picking a model without your own test is a bet.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
