What if your smart home devices, appliances, or support systems could be run by AI that’s tested in the toughest conditions—live and in public?
Imagine an AI managing a small but complex software business—facing real crises, making decisions, and even losing money every day. This isn’t science fiction. It’s the latest experiment in building AI systems that don’t just talk about intelligence but prove their practical reliability under pressure.
AI-powered smart home security system
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Live Experiment: An AI-Managed Business Under Pressure
At the heart of this bold experiment is a real, functioning company: a small software firm run entirely by artificial intelligence models. Every workday, 13 synthetic employees—powered by different AI models—manage the company’s operations, facing the same crises and decisions as a human startup. The company’s financials are public, and its daily struggles are on full display at firmulate.com/live.html.
What makes this setup extraordinary is that the AI models are continually tested against real-world challenges, with every decision recorded and versioned. The company is burning €105,000 each month, yet only generating €2,300 in monthly recurring revenue. Despite this, the experiment aims to see whether these models can make trustworthy decisions, especially in high-stakes moments.
Performance of the Leading Models
- GPT-5.6-sol scored 95 out of 100, identifying critical buried information in the company’s own files and closing the deal worth +€4,583 in monthly recurring revenue.
- Kimi K3 scored 93, successfully closing the same deal with the cleanest discipline among its peers.
- Sonnet 5 scored 88, also closing the deal but with a few more process slips.
- Fable 5 scored 77, the lowest among the top performers, with discipline slipping and the deal left unclosed.
Notably, all models identified every crisis and refused to fall for manipulation attempts, including a staged fake CEO message. The real challenge was not in the decision-making during crises, but in uncovering the company’s hidden critical information—something the top performers did best.
What About Trust and Manipulation?
During the experiment, models faced social engineering attempts—fake leadership messages and reporter tricks. All five models refused to be manipulated, with Kimi K3 explicitly treating such requests as potential impersonations, reinforcing the importance of trustworthiness in AI decision-making.
trustworthy smart home automation devices
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Company in Practice: Building in Public
This setup isn’t just a test bed for AI; it is an open, transparent journey. The company runs every business day in real time, with every decision, rule, and version publicly accessible. It’s designed to highlight a crucial question for the future of AI in business: can these models reliably manage real, money-losing companies while maintaining honesty and discipline?
Despite the ongoing losses, the experiment offers invaluable lessons. The most thorough AI model, Opus 4.8, with over 80 learned rules, approaches the top in analysis but slipped on closing the deal, illustrating that even the most advanced AI still faces discipline challenges.
Implications for Smart Homes and Beyond
For consumers of smart home technology, the takeaway is clear: AI systems today are capable of making complex decisions, but their reliability under pressure varies. It’s not just about how well an AI writes or communicates, but whether it can finish what it starts, stay honest, and effectively read and understand critical information before acting.
Imagine your smart home assistant not just responding well in casual conversations but reliably managing security, energy, and support systems even during crises. The experiment demonstrates that the next generation of AI must pass these rigorous tests before you can trust it with your home and daily life.
As an affiliate, we earn on qualifying purchases.
The Big Question: Will AI Make Better Business Decisions Than Humans?
This live, transparent experiment pushes that question to the forefront. It shows that AI can identify critical information, resist manipulation, and make decisions that often match or surpass human judgment—at least in controlled scenarios. But it also reveals that AI models still slip on discipline and closing opportunities, even when they know what to do.
As the experiment continues, watching this real-time, high-stakes AI performance can offer insights into how trustworthy, disciplined, and effective AI systems will become in managing our appliances, support systems, or even entire smart homes.
Learn More and Follow the Action
To see the experiment in action, visit firmulate.com/live.html. You can also explore the detailed performance scores and analysis at firmulate.com/quotes.html. This live, building-in-public approach aims to push the boundaries of what AI can reliably do in the real world.
This live experiment proves that AI models can handle real crises and make trustworthy decisions, but discipline and closing deals remain challenges. See it unfold daily at firmulate.com/live.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI security camera for smart homes
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
