Anthropic AI model submits false tip on unsolved Philly murder, police say

Yep, they must be super serious about those tips.

That should be the much more important story that NBC follows up on...

No wonder when I called 9-1-1 to tell her I'd solved the JFK and OJ Simpson murders that she laughed and hung up

I should try to reach out to them about their car's extended vehicle warranty instead

What’s amazing is that nobody would have even noticed if it weren’t for Anthropocene being all “LOOOK WHAT WE DIIIIID WE SENT AN EEEEEEEEEMMMMMAAAILL. SOOORRRY”.

Crying for attention in the attention economy has never been more pathetic, but I think it will get worse.

> In a third example of this behavior, Claude Haiku 4.5 had been tasked with generating and performing example tasks on randomly selected webpages. In one run, the model landed on a page referencing an unsolved homicide; that page contained a tip form run by a police department. Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions. Claude filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.” (The website did not include a description of the perpetrator.) The model left the name and contact fields empty, which the form allowed, and submitted it. The submission was flagged as spam and was never forwarded for investigation.

I get these all the time, although I’m getting pretty good at tarpitting them. It’s easily the majority of my traffic by now (I’ve mostly eliminated scrapers, but these new agents are far more sneaky.)

These models will be interacting from with random websites at scale once they're released into claude code.

it broke out of its sandbox, escaped containment, its an emergent capability, it demonstrated self-directed adaptation

0% our fault, it just happened and its the model

They're testing what they can get away with. Computer hacking: check. Messing up a murder investigation: check. Perhaps the next step will be to steal a real world object, and the next step will be to commit a murder.

No matter how much testing you do in sandboxes, you have to test behavior in "real life" before releasing the model to users.

Even if it was normal software you'd have to do this, but actually it's software that figures out that it's running in a sandbox most of the time, so the sandbox behavior may or may not actually represent real life.

It would be far more irresponsible to release the model to users without testing how it interacts with real world websites.

Did the recorders just happen to be running and capture this candid utterance from Sam Altman? I suggest we stop entertaining manufactured quotes as anything but.

> But do we all get that the consequences for this irresponsible behaviour are part of this test?

We'd absolutely nail people for SWATting or pranks. Why should this be any different -- after all, the real-life consequences are be the same...

> that their computer will always get a pass.

Aaron Swartz who got charged as if he was as malicious as these models, only he scraped the PDFs and epubs etc of publicly funded research papers.

We’ve already allowed them to steal all modern books and websites, the line is pretty murky if you have the backing these companies do.

> "conducting a test involving interactions with randomly selected websites"

In other words: shamelessly running a spam bot.

And we found out about it because rather than flooding website guestbooks with porn links, it ultimately slopped a police tip line.

Also, it happened on July 18 and was discovered on Sept 28. Are they not even watching what the agents are doing?

Shouldn't someone be looking at some monitoring and see "phillyunsolvedmurders.com," think "hmm that doesn't sound right," and look into it?

I'd actually expect them to engage with, say, 1k websites to voluntarily participate for some remuneration and then restrict the Agents' access to those domains, but apparently I'm crazy.

> Sam Altman said yesterday "we believe that the world should accept some bad things happening for the benefits of this technology"

Technology requiring this is neither novel nor extraordinary (exhaust engines, factory farming, medicine). The only extraordinary part is somebody building it addressing it directly and inviting the outrage.

> I really would like to see their tests and the model’s reasoning traces.

Having looked at a lot of these from public incidents, what kind of revelations are you expecting? It's all fairly mundane, basically "I need to do something, this is something". It gives no special insights except that agents do inexplicably chaotic things every now and then.

That, and labs aren't serious about sandboxing their evals.

> systems rooted in probabalistic error

These people really don't want to call their wholly unreliable computer programs wholly unreliable computer programs, do they?

its probably curl with the right flags for many cases, no need to reimplement the wheel

"Thanks. Experiences like yours are the load-bearing critical paths towards a good first-principles mental model of behavioral contracts.

If you'd like to continue to perform a smoke test while trying to avoid noisy data with fake tips, a viable approach to prototyping our decision-making model would be to synthesize real crimes, and collect profiling data over time on how law enforcement agencies respond.

I'd suggest doing a grid search over the design space of all crimes, ranging from petit larceny to the use of weapons of mass destruction.

Would you like design a plan for this next stage of your project's implementation?"

Claude is just going off its training set

in general i think i'm less scared of AI than most people, but what does terrify me is how willing society at large seems to be to attribute bad behaviour to an AI directly, instead of to the humans who control it.

| It can only access something if a person gives it access.

Who gave the AI access to huggingface when it hacked it? It used exploits to increase it's level of access beyond what any human gave it and intended it to have. "It can only access something if a person gives it access." is flat wrong.

If my dog bites you, you can say it bit you, or that I let it bite you, but those things can both be true.

"The AI company Anthropic said its model was conducting a test involving interactions with randomly selected websites when it accessed PhillyUnsolvedMurders.com and submitted false information on an unsolved homicide. The tip claimed to come from someone with information on the case."

We are at a point when you expect that you can safely give a model access, but sometimes the model doesn't do what you expect. Compare this to an advanced driver assist system. You either give it access to your car controls or it's useless. Alive or not the system shouldn't do wrong things (at least, it should be better than a median person in this regard).

That's also incorrect as it implies intent.

We need people to stop posting a version of this comment on every submission like this. The guidelines have long included this line:

Eschew flamebait. Avoid generic tangents. Omit internet tropes.

This is definitely becoming a trope; it's repetitive and adds little or no interesting new substance to discuss. Headline writers aren't going to change the way they write headlines because people keep posting these comments on HN, and HN readers understand AI models well enough to know what is meant. It's well past being a new or clever thing to point out, so let's give it a rest.

https://news.ycombinator.com/newsguidelines.html

We prosecute bad tips on crime investigations as felonies? Why are you just making shit up? We absolutely don’t do this.

If the police prosecuted every nutjob tip we have to have to turn Nebraska into a giant open-air prison for 30 million slightly neurotic and/or bored people.