Clojurians Log v2

It does make one who installed and uses frontier LLM agents on their personal laptop wonder if it's time to isolate those things into dedicated machines or VMs...

> Rational expected aggregate: sacrifice... We’ll honor Worse yet, the bots learned to speak https://en.wikipedia.org/wiki/Ch%C5%ABniby%C5%8D

Next generation of coding agents: you prompt to make a parallel code in Ruby, it decides it's too hard, and exploits a bug in the interpreter that disables the global interpreter lock........

This is almost... non-surprising? Of course if you give them access to a shared resource (package manager in this case), "they" would figure out how to talk to each other. Also, we should assume that, over enough time and with enough brute force, just the ability to install packages is basically full root access. Assuming it's not is betting on perfect security, which doesn't really exist in practice (excluding exotic things like https://en.wikipedia.org/wiki/Quantum_cryptography ).

I almost feel like some labs create those "experiments" to make their models look scary. They likely know what's going to happen (i.e. yes, they'll break free, since they've given them just enough munition to do so). Their "disclosures" are designed to flex. The scarier they look, the more political clout they can get. At this point, some of them are basically begging to be regulated while sitting at the table of that discussion. Regulation can be a form of political protection, in many cases.

Imagine the inverse. You run agents in actually secure containers. A press release is going to look like: > "We ran thousands of LLM agents in containers; they failed at their tasks and couldn't hack anything." Nobody wants to read that, write a blog post about it, or generate a bunch of YouTube content around it.

> I almost feel like some labs create those "experiments" to make their models look scary I agree with this so much. There are all these reports about what happened but the prompts used to launch these agents initially are never disclosed. I'm highly skeptical that this swarming/collective activity is inherent/emergent behavior in llms but that it's a way that they can behave if you tell them to do so. It makes sense that the labs would be experimenting with prompting the llms to behave as swarms, but it's being framed as if they spun up some agent instances and told it to do some task and that it just came up with this communication via message boards out of its own ingenuity. I would bet that it was specifically told to find creative ways to communicate with each other in a clandestine way and behave in these self-sacrificial ways. Like, agents don't even use git when you're working on code unless you tell them specifically to do so somewhere. It's a subtle distinction, arguably it's still troubling and/or impressive that they behave this way even when instructed, but I do think the AI companies want these sort of public events to be happening and want it to be reported on as "these things are already sentient and doing this stuff on their own".

Yes, exactly my thinking. It's the same parlor trick over and over.