"My work in Redmond had to do with what we might call the vN immune system," Sarton said. "I hypothesized that bluescreen events were really failsafing events in disguise – that for whatever reason, the iteration in question had begun to failsafe, but gotten stuck."
"So you know all about broken failsafes," Amy said.
"I thought I did. Then you came along." Dr Sarton made kissing sounds at a large lump of red silk cushion in the corner, and it promptly inched its way across the floor and curled around Amy and Javier's ankles. "Have a seat."
They sat. The cushion was warm, and it purred slightly. "Please don't mind the stains," Dr Sarton said. "I sometimes sleep here."
Javier muttered something behind his hand that sounded a lot like "Chimps…"
"The vN immune system is comprised of two parts," Dr Sarton said. "The first is exterior – your body remembers what it should look like, and edits out the damage that occurs. The second part is interior. You're protected against a wide variety of worms, memes, viruses, and so on. Most of that is unnecessary because you're not actually wired to a network. Connected models were considered, early on, but there was a very serious concern that if and when the failsafe failed – as the result of a hack, or a virus, or an emergent property – the failure might spread virally among vN."
"Is that what happened to me when I ate Portia?" Amy asked. "She hacked me from the inside?"
Dr Sarton licked his lips. He steepled his hands. "I'm not sure." He made a pinching motion at the display, and new footage opened up. Amy recognized her clade immediately. Groups of identical women shuffled in and among triage units full of wounded humans, squirting skin glue on wounds, holding hands, administering fluids.
He said, "After the Cascadia quake, your clade was crucial to the relief effort." He pinched again, and the view bounced downward, toward one tent where a man with ruined feet screamed silently at the camera. His nurse ignored his noise, but kept working diligently on removing glass and other debris from the wreckage of his feet. "This was before you all attained sentience, of course. The nursing vN were easy to program and deploy as search and rescue units. Several were lost in the aftershocks. They were spread quite thinly; a Japanese prototype of a networked model was sent over to assist, but their container was lost in the Pacific."
Amy looked over at Javier. He kept his gaze carefully pointed away from the screen. "Change the channel," Amy said.
Sighing, Dr Sarton did so. "I know that it sounds very trite, but your foremothers did a lot of good in the past. When drug-resistant bacteria infected hospitals, for example, they could go on working and treating patients for days at a time without rest or demands for overtime pay – or any pay at all, for that matter. Or if healthcare wikis were hacked or wiped or went down for any reason, your model accurately remembered even the tiniest details related to individual patients and their treatment history. There was a time when no clinic in this country was complete without one of you."
"Then what happened?" Amy asked.
"You got smart," Dr Sarton said. "And like all underpaid workers who see a better opportunity elsewhere, you left. But you left with a profoundly different failsafe than the other vN."
Amy thought of Javier's turned-away face, his closed eyes. He couldn't even watch the footage of the nursing models at work. There was too much hurt and suffering going on. But the nursing models had lived it. They had observed that pain and treated it and gone on about their business. It was their job.
"They made us this way," Amy said.
Now you get it.
"Your clade's failsafe was already destabilized by the time you attained self-awareness and the ability to iterate." Dr Sarton crossed his legs and leaned back in his chair. "For most vN, it's a question of stimulus response. Their failsafes are reactions to perceived phenomena. For your model, the failsafe lay along a specific decision pathway – the decision to hurt a human being. Your model could monitor suffering so long as they were striving to alleviate it, but could not make the choice to cause it. They could never amputate a limb without anaesthesia, or break an infant's collarbone if it were tearing its mother's birth canal apart, even if the wound were causing her to bleed out."
Beside her, Javier tried to settle himself deeper into the cushion. His fingers knotted in the material. His jaw was tight. "OK, I get it," Amy said. "We were different. Our failsafe was weak. Eventually it broke all the way, right?"
"That's the prevailing theory, yes," Dr Sarton said. "No one knows exactly when it happened, or where, or for what reason. Portia can probably tell us more."
Tell him you morons brought it on yourselves, she said. Tell him you begged us for it.
Amy sat up taller. "I don't want her version of events. I want to get rid of her."