First off, the parties involved are among the most tech- and computer security-literate organizations on Earth: the AI lab itself (OpenAI), an elite software evaluation organization (METR), and the company that hosts half the world’s open models (HuggingFace). Yet, while the incident with OpenAI’s model going all Spring Breakers with the Hugging Face townies may have involved some of the ominous-sounding risks we’ve come to know—deception, misalignment, “going rogue”—the developer’s own telling admits that more diligent (but still basic) human monitoring would’ve caught it weeks earlier. Same for Anthropic, which, if it weren’t for taking notice of its rival’s incident, wouldn’t have caught its own rogue model (which the company did, only months after it happened).
Second, in OpenAI’s case, the failure was not exactly a model going astray. It was a failure to configure a testing environment correctly, according to globally accepted but voluntary standards. In particular, ISO/IEC 27001: 2022 Annex A control 8.31 and 8.22. According to respected cybersecurity expert Tony Vizza, “if your red-team range has a routable path to the production internet through a vulnerable internal proxy, the problem predates the AI.” Basically, OpenAI’s model broke free of its testing environment not by finding a new way to pick the lock, but because the testers failed to deploy the relatively old-fashioned locks long known to cybersecurity experts.
Or put another way, offense advances faster than defense. Not because of the technology itself, but because the humans running it don’t ever keep up. It’s natural to race ahead at full speed to push the capability frontier, while simultaneously forgetting to do the mostly mundane things we collectively refer to as security (information security and cybersecurity). Think of it as the flip side of the AI “verification tax”—the “extra human effort required to review, correct, and validate AI outputs”—that’s paid by end users.
For Anthropic, a “misunderstanding” led to a misconfiguration. No one skimps on the exciting new capabilities, but meanwhile Stanley in IT can’t get his budget approved to hire another information security specialist.
0 comments
log in to comment.