The recent rapid advance in the capabilities of artificial intelligence (AI) applications I think qualifies as a disruptive technology. The term “disruptive technology” was popularized in 1997 by Clayton M. Christensen. To summarize, a disruptive technology is “an innovation that fundamentally alters the way industries operate, businesses function, or consumers behave, often rendering existing technologies, products, or services obsolete.” AI is potentially so powerful, and changing so quickly, that it is challenging to optimally regulate it. We are caught in a classic dilemma – we do not want to hamper our own competitiveness in a critical new technology, but we also don’t want to unwittingly create new vulnerabilities or unintended negative consequences. For now we seem to be erring on the side of not hampering competitiveness, which basically places us at the tender mercies of tech bros.
Which is partly why I found the conflict between Anthropic and the Department of Defense (still the legal name) so fascinating. In short, Anthropic’s powerful AI application, Claude, has at least two significant internal “red lines” or guardrails – it cannot be used for massive domestic surveillance, and it cannot be used for final military targeting, without a human in the loop. Anthropic CEO Dario Amodei has not backed down on this – he says that the first restriction on domestic surveillance is simply a matter of ethics. The second restriction, however, is mainly a matter of quality control – their system is still vulnerable to hallucinations and is not reliable enough to count on for final targeting decisions. Hegseth has criticized his concerns as “woke” and a critical vulnerability for the US military. More charitably, he say essentially that the US military is using the application lawfully, and should not be restricted in any lawful use of the software. Others have also stated that in an emergency they have to know the software will do whatever they ask it.
This conflict has many deep implications, and is beyond what I intend for this blog post. What I want to focus on is the fact that an AI application is creating this ethical dilemma, and forcing us to ask – who should control such awesome power, the CEO of a tech company or the Federal government? It seems that we are facing or about to face many similar questions provoked by the disruptive nature of recent AI applications.
Anthropic, in fact, is at the center of another similar discussion, involving the security of the internet. They have a new application, Mythos, which is an AI coding app. Mythos is potentially disruptive in two ways. The first is more mundane, and certainly not unique to mythos – it allows for non-coders to do what is called “vibe coding”, giving an AI coder a natural language description of the application you want, and the AI coder making it. Why this is disruptive is because it takes coding out of the limited hands of a relatively few highly trained and skilled individuals and puts it in the hands of everybody. This can lead to the proliferation of code that has not gone through any rigorous safety testing for vulnerabilities.
But the feature of Mythos that has many experts (including those from Anthropic itself) very concerned is that the program turns out to be excellent at identifying security vulnerabilities in code. I mean – really good. It has found vulnerabilities that have been sitting there unnoticed for years, and can reliably exploit them. When Anthropic realized how good their software was at essentially cracking software security, they had an “Oh, shit” moment. We are at an “inflection point”. Anthropic estimates they are 12-18 months ahead of the competition, so very soon similarly powerful software will proliferate. If we do not lock down critical software infrastructure by then, the internet can be screwed. Much of the internet and many applications run on core software that is open source, maintained by volunteers with shoestring budgets. Mythos has already cracked open some of these core bits of code.
Turning the internet, and essentially the software infrastructure that increasingly runs our world, into a cybersecurity nightmare is, I would imagine, not good for business. So Anthropic has given a preview version of Mythos to a consortium of 40 software companies, including their competitors, to basically give them a head start in finding and fixing any vulnerabilities in their software (which they are calling Project Glasswing). They are also dedicating some money to fund the project, especially for open source software. This all sounds great, and maybe this will fix the problem. Hopefully we will eventually see this as a Y2K situation, the disaster that never happened because we prevented it.
What this affair highlights is how the disruptive nature of AI is creating the potential for significant problems, if we do not stay ahead of it with rational regulation and quality control. It seems that Anthropic is trying to be an ethical and responsible corporate citizen, and that it recognizes the power of its products. Thank goodness for that – imagine if the same tech were in the hands of a less scrupulous or responsible company? It’s pretty easy to imagine. This is happening at a time when the Federal government not only has no apparent interest in regulating AI, they are trying to prevent the states from doing so either. And they are throwing a temper tantrum when they cannot use their new toys without restrictions.
Going forward we should not rely on the noblesse oblige of tech CEOs. We need to make sure that security and ethical restrictions are baked into any new applications. I am all for vibe coding, for example, but such apps need to have rigorous quality control, so we don’t fill the world with the coding equivalent of AI slop, creating a vulnerabilities tsunami. Perhaps this consortium of tech companies will evolve into something bigger – an organization dedicated to safely and securely developing this technology. This means, of course, we need to get buy in from China, which means we need international standards to regulate this tech. I think of it like nuclear weapons. AI is a very different kind of threat, but it is also a powerful technology that would benefit from international agreements so that we don’t accidentally destroy our civilization.
