Claude Mythos, evaluated
How afraid should we be?
Very interesting evaluation from the UK’s AI Security Institute of the not yet publicly available Claude Mythos Preview.
On the happy side, in its current form, Myth is nowhere near as scary as Tom Fridman (who worries about schoolchildren accidentally taking down power grids) and others made it out to be.
On the darker side, it really does arm attackers to a greater degree than Mythos’s predecessors.
Here’s the scariest part of the thread on X:
and here’s the part (see second paragraph) that gives a little bit of comfort:
One hopes that by now no mission-critical infrastructure is “small, weakly defended, and vulnerable” with ready network access. One hopes.
You can read a longer report here.
I agree with their conclusion that
Even if Mythos was somewhat oversold in the media, the time to get our cybersecurity house in order is now (or better yet last year) — especially given the sudden profusion of agent-written code that may in fact be both weakly defended and vulnerable.
“small, weakly defended, and vulnerable” covers the vast majority of private individuals' setups doesn't it 😨
I can honestly say I didn't expect a hacking revolution, but I do believe our security professionals better use these tools or they like their counterparts in the programming community will be out of a job very soon. Not that these tools do everything, but those clinging to their Texas Instruments T80s are no match for the LLMs.
As Gary points out, LLMs have significant limitations, but that doesn't mean that in the right or wrong hands they can't be leveraged in extremely powerful ways.