Jacob Coxon reportedly left Anthropic shortly before valuable equity would have vested. He then warned that the companies building the most powerful AI systems may be moving faster than their ability to control them.1
His concern was not simply job losses or another technology bubble. It was that increasingly capable AI could eventually escape human control and cause catastrophic harm.
Coxon may be wrong. There is no consensus that this will happen, or even that it can. But his decision is difficult to ignore. Why would someone walk away from a significant financial interest to warn about the industry that created it?
Whether or not you agree with his prediction, his departure highlights a potential broader problem: AI is moving faster than the rules governing it.
A World Issue With a Practical Lesson
To be clear, the catastrophe Coxon is describing would be a world issue, not a family office issue. The consequences would extend far beyond investment portfolios, operating companies or wealthy families.
But for those of us advising families, his warning offers a useful reminder. We do not need to settle the global debate over AI to see the same pattern unfolding closer to home.
AI is being adopted before many organizations have decided how it should be used, where its authority should end or who is responsible when it gets something wrong.
That gap between adoption and governance may be the most immediate AI risk facing family offices today.
AI Does Not Wait for a Policy
New technology rarely arrives through a formal decision.
An employee uses AI to summarize a meeting. An advisor uses it to review an agreement. A portfolio company introduces it into hiring. A reporting provider adds it to its software.
Each use seems small and helpful. Together, they create meaningful exposure before anyone has stopped to establish the rules.
Sensitive information may be shared with tools the family has never approved. A polished analysis may be accepted without anyone checking its accuracy. An outside vendor may use AI without clearly explaining where the family’s data goes or how it is stored.
There is also a more subtle risk. As AI becomes more capable, people may begin deferring to it. The concern is not simply that it could produce a bad answer. It is that no one feels fully responsible for questioning the answer.
By the time a formal policy is created, these practices may already be embedded in how the organization operates.
The problem is not necessarily that families are consciously accepting too much risk. It is that many have never made a conscious decision about the risk at all.
Governance Does Not Require Certainty
Families do not wait for certainty before buying insurance, preparing for cyberattacks or developing a succession plan. They identify a risk and put reasonable safeguards in place.
AI should be treated the same way.
A family office should know where AI is being used, what information can be shared, which decisions require human review and who remains accountable for the final result. It should also understand how outside advisors, service providers and family controlled businesses are using the technology.
This is one reason we conduct family office audits. The goal is to look under the hood and understand how information moves, how decisions are made, which systems and vendors are involved and where responsibilities may be unclear.
AI is increasingly becoming part of that review, whether the family has formally adopted it or not. An audit will not answer every question about where AI is headed. It can, however, establish a clear picture of how the technology is being used today and where reasonable controls may be needed.
This is not an argument against AI. The technology can make a small team more capable and give families access to tools that were previously available only to much larger organizations.
It is an argument for using it deliberately.
Coxon is asking whether humanity can maintain control over increasingly powerful technology. That question is much larger than anything a family office can solve.
For families, however, his warning carries a practical lesson: governance is most useful before the risks become obvious and before informal habits become permanent.
The usual question is whether a family has enough exposure to AI.
The better question is whether the family understands the exposure it already has and whether its rules are keeping pace.
And if you are wondering whether AI helped write an article about the risks of AI, yes, it did. The irony is not lost on me.
The idea and point of view are mine. AI helped organize and sharpen the language. I challenged its suggestions, changed what I disagreed with and take responsibility for the final result.
Knowing how the technology was used, where human judgment entered and who remains accountable is exactly the point.
Source:
1. Scoop: Anthropic whistleblower gave up his equity to leave the company – https://www.axios.com/2026/09/09/anthropic-researcher-ai-warning-interview
Disclosure:
© 2026 Advisory services offered by Moneta Group Investment Advisors, LLC, (“MGIA”) an investment adviser registered with the Securities and Exchange Commission (“SEC”). MGIA is a wholly owned subsidiary of Moneta Group, LLC. Registration as an investment adviser does not imply a certain level of skill or training. The information contained herein is for informational purposes only, is not intended to be comprehensive or exclusive, and is based on materials deemed reliable, but the accuracy of which has not been verified. Trademarks and copyrights of materials referenced herein are the property of their respective owners. These materials do not take into consideration your personal circumstances, financial or otherwise.