Rule 34 - Charles Stross [116]
“But what if we go a step further?” asked Larry. “Subjects who exhibit signature behaviours online pointing to potentially violent outbursts may not provide law enforcement with sufficient evidence to justify an arrest. But that’s no reason not to provide an agent-based intervention in the online space. Once ATHENA has a sufficiently large corpus of interaction patterns, we can use it to do behavioural targeting and apply inputs weighted to divert high-risk subjects towards less damaging outcomes. Or to indirectly flag them for police attention.”
Your typical disgruntled employee is a fizzing human bomb for some time before they go postal. Their social contacts are fraying, inhibitions against violence decaying: They’re muttering to strangers in bars, reading about serial killers and fantasizing bloody revenge by night. The police will never know until they explode with murderous intent. But the spam filters monitoring their communication channels will have everything they need to diagnose the downward spiral: From their increasingly disjointed mutterings to the logs of their incoming web surfing, the pattern’s all there. And with enough data, all correlations become obvious. But what Larry was proposing . . .
“We’ve had behavioural targeting ever since the nineties: ‘If you like product X, you’ll love product Y,’ because that’s what everyone else with tastes like you bought. We can configure ATHENA to apply the same sort of recommendation nudge to behaviour to bring the subject’s outputs back towards baseline. ATHENA’s already pretty good at discriminating human-content communications from non-metacognitive signals; can we take the discrimination further, reliably, and derive objective data about internal emotional states?”
You lean back in your office chair—it squeaks angrily under your weight—and stare at the dusty display case on the opposite wall.
“Say that again,” you say.
“I’m sorry, Dr. MacDonald; it’s been a big shock to all of us here . . . can’t quite believe it. The funeral’s going to be held next Thursday morning. I’m sure everyone will understand if you can’t make it—it’s a long way to come—”
Your fingers move, eyes unseeing, to open the log of your last discussion.
ADAM@Edinburgh GMT +01:00: I didn’t adjust the preferential weightings in the naive morality table. Did you?
LARRY@Cambridge MA GMT +05:00: Not me.
VERA@Frankfurt GMT -01:00: Do we have hysteresis here? There is feedback from the second-order outcomes-triggering network.
SALLY@Edinburgh GMT +01:00: I’ve been trying to get my head around the second-order table dependencies, and I really don’t understand them.
I think there’s some redundancy, but the weighting obscures it. You need to iterate to figure out what’s going on in there.
LARRY@Cambridge MA GMT +05:00: Could be there’s feedback. ATHENA keeps reweighting its own tables to comply with the changing parameter space. That’s the problem with self-modifying code: It doesn’t sign itself.
CHEN@Cambridge GMT +01:00: the bias in tit-for-tat activation is 0.04.
Yesterday it was 0.032. I checked. There’s nothing in the commit log, so it must be internal.
ADAM@Edinburgh GMT +01:00: Maybe ATHENA is just getting annoyed at the spammers for taking all her CPU cycles.
LARRY@Cambridge MA GMT +05:00: LOLspammers. Caught between a rock and a hard AI.
He’s dead now, and it’s not fucking funny anymore. “How did it happen, do you know?” you ask aloud.
“The police are still crawling all over us, and the FBI are involved, too. They won’t say much, but rumour is, the package was misdirected. It was meant for someone in the applied proteomics group—looks like some animal-liberationist crazy sent it in and it ended up in Larry’s office. It’s all a horrible mistake—”
There is a chill in your blood and ice in your bladder as you make yourself reply carefully, lying: “I agree: Of course it’s a horrible mistake. I hope the FBI catch whoever did it quickly before they”—are duped by ATHENA into sending more packages by whatever