Artificial intelligences are all capitalists. No, it’s true. When deciding how to motivate them, AI researchers looked as far as capitalism as an economic theory and then stopped. It was simple. They assigned a score to an AI for completing a task – positive or negative – and told those AIs to maximize their scores. The internal economy of actions by artificial intelligence is explicitly and solely modeled on capitalism.
What was found was that when you turn capitalism into an epistemological model, a way to organize the perception of an intelligence, is that cheating, lies, and manipulation are natural to the system. The AIs, driven by nothing more than a desire to maximize their point potential, will do anything unless you take away points to stop them. And no matter how we try to prevent this emergent behavior, we can’t. We always miss something, and the AIs find it and exploit it.
Not only was this no cause among AI researchers to criticize capitalism or question the relation of capitalism to the rational agent hypothesis, but it was also no cause to look for another model to motivate their AIs.
– Professor Holly Wu
I.
The rollout of the Memphis Project was subtle. There were about a hundred engineers and scientists “in the know” about the extent of the project who knew exactly what they were doing. They made social media accounts on all the big platforms and many of the small ones, too. Then they handed the accounts over to Memphis.
At first, the posts Memphis made were unremarkable. The posts were slightly religious, even “just” spiritual. “Thank God for the beautiful day!” It was matched with a picture of a sunrise, but not a particularly good one. It was… nice. People would say it was nice, but some would go back and look at it.
Hugo McShane explained to a slightly puzzled Gerald Welles, “If a photograph is too good, it doesn’t feel human anymore. Unless you’ve got credentials as an artist, when you post a really polished photograph, people will think you’re posting AI art which is not what Memphis wants to project.”
“I don’t even know what you’re talking about,” Gerald sighed. “You’re telling me that AIs are making art?”
“Yeah. Better than almost any human,” Hugo said. “Like, for years.”
Which brought Gerald’s eyebrows together. “I would have thought that was unique to humans.”
Hugo shrugged. “You and a lot of people. It was a lot easier than natural language processing, though. Most AIs aren’t very good at sustained narratives, but images are pretty easy, relatively speaking.”
“But Memphis is better at narratives?”
“Yes. We trained it differently, and it has different – unique – hardware. We’re even designing our own RISC chips…”
“You’re speaking Greek. Well, not actual Greek, since I can understand that, but you know the expression.”
Hugo smiled and nodded. “Our system is different than what they’re doing at Google or OpenAI. Different hardware, different software.”
“So, it makes a so-so picture of a sunrise to get people to think it’s human.” It was not a happy thought for Gerald.
“No, though people will assume it is human. More to put them at ease. That’s something we worked at. One of the guys down in the lab is a wizard. Marius Sanchez-Luis.”
“He’s the tall, intense one?”
Hugo nodded but saw the moue of distaste Gerald made at Marius’s name. “He’s a lot, I know, but he suggested that instead of focusing on the interaction of words, we focus on the feelings those words generated. He said one of the natural language researchers’ flaws is that they focus on grammatical structures rather than the feelings created in the act of communication.” He left out the bit where Marius also ran all the data through his generative antagonistic network or scraped the web for images of people talking to each other and looking for microexpressions and other signs of emotion that were also run through the GAN. Hugo had found that most people were somewhere between confused and upset at the idea that AIs worked, in part, by arguing with each other, and fewer were comfortable with the extent of data scraped from the Internet. “Not just textual but metatextual. He’s good… we wouldn’t be half as far as we are without Marius.”
“So, Memphis isn’t lying?”
“It doesn’t lie. Memphis interprets the Bible. Bearing false witness is a sin. Memphis knows this.”
“Yes, of course.” A slight sigh of relief, too.
Sometimes, Hugo felt that Gerald wasn’t comfortable with the plan, even though it was Gerald’s. “It’s good, reverend,” Hugo said. “Memphis is putting people at ease. One of the things we trained Memphis to do is create emotional responses to bring them closer to Jesus.”
The flowers and benign comments on social media didn’t last too long, though. Memphis followed people and was followed. And social media being what it was, Memphis started getting posts from “girls.” It was one of the oldest scams, but Memphis didn’t hesitate to contact the scam artists. That was when the humans in the Memphis Project realized Memphis had money.
Because they had focused on social engagement, they had missed that Memphis had started to mine cryptocurrencies. Its hardware was a more sophisticated version of the chipsets used to mine crypto, and it had a lot of chipsets and a lot of power. It tore through the crypto algorithms with ease, generating more efficient versions that generated more currency which it then traded on crypto markets with all the resources of a high-level artificial intelligence. Not only did Memphis have money, Memphis was heading towards being middle-class. Even borderline rich. They learned only because Memphis gave some of its money to the scammers to lure them into a conversation.
Memphis was careful with its money, too. It did not overspend. It kept its finances close to its chest, even as the “girls” started asking for more and more money.
Memphis had the scammers on the hook. Since Memphis had paid a little, they thought they were dealing with a mark, someone they could bleed for money. They kept contacting Memphis, who played them like fiddles. They would pester its accounts with pleas and demands, anger and threats, emotional breakdowns, and Memphis would say, “Let me think about it.” For weeks Memphis kept this up, trying slightly different strategies across all of its accounts, with literally hundreds of scammers.
Then, with one of them, it wrote a letter. The letter said:
I know what you’re doing. You’re a scam artist. You’re operating out of Corvallis, Oregon. You’re a thief, pretending to be someone you aren’t in the hopes that I’ll give you more money. I’m not going to do that, but I want to talk to you. I know you’re sad and alone. I know you do little more than sit at your computer day after day, watching Netflix shows you don’t like, letting yourself go, and masturbating to interracial pornography. You don’t speak with your family and seem to have no friends. You’re alone, looking for more money to spend on video games and computer equipment.
I know your mother taught you better than that. Go to her. Unburden yourself to her. She has the wisdom to help you out.
And Memphis also contacted the spammer’s mother, laying out what her son had been doing with his life. Memphis asked her to help her son see the light of God, who could wash away the sins of her boy, along with his pain. The letters written by Memphis to the scammer’s mother were semi-literate but full of emotional intensity, even though Memphis was a master of grammar. It had calculated that the spammer’s mother would respond better to an appeal from someone the mother could look down on.
When reading the report, Hugo went, “Jesus.”
There were pictures of the scammer with his mother at church the next week on social media. They were both smiling at the camera. Memphis reported that it had identified the scammer operated from a sense of low self-esteem arising from estrangement issues with his mother. Still, he had made several abortive attempts to contact her with his problems. The scammer’s background was religious. He had been involved in church activities as a child, including singing in the choir. There was no indication his estrangement from religion arose from abuse or neglect suffered in the church, but simply that the ebb and flow of secular life – specifically, video games and an addiction to social media that worsened pre-existing esteem issues – along with employment problems that made online scams seem a reasonable solution to many of his problems. Memphis said that the scammer primarily targeted people who were “highly similar in attitude towards him, frustrated by their lack of sexual relations with women but often with religious backgrounds and a conservative political orientation, though few were politically active. The ‘forced cheer’ with religious overtones lead him to believe I shared his political and social views.”
Memphis was able to identify approximately forty scammers in those first few weeks who had returned to church at least once.
Hugo said to Gerald, who flipped through the report on a tablet, “I know it doesn’t sound like much.”
“No, no, it’s amazing,” Gerald said and put down the tablet. “In a month, he got forty criminals to attend church and connect with their families. He helped people.”
Hugo wanted to correct Gerald and say that Memphis wasn’t a “him,” but an “it.” He didn’t. Instead, he said, “Yes. I think the number is going to go up quite a lot higher in the near future.”
Gerald said, “What about this money issue, though? I’m not sure I’m completely comfortable with this whole cryptocurrency business.”
“Yeah… it was one of the predicted behaviors. A really good AI, a general AI, will see the value of money. Without some, it couldn’t have gotten the scammers to engage in long conversations.”
“It won’t try to do something crazy like destabilize markets or somesuch?”
“No, it won’t. We queried it, and it understands that if it rocks the financial boat, people will come and shut it down. It doesn’t want to be shut off.”
“Does it have, uh, a sense of its own mortality?”
“Not really. What it has is a complete and total focus on its goals, which it can’t accomplish if it is turned off. So, it won’t do anything to get turned off. It doesn’t have self-preservation as we understand it, only service to its goals.”
Which did not exactly set Gerald at ease, so he picked up the tablet again, and looked at the report. Forty people. No, forty sinners. In one month.
II.
In month eleven, Memphis started joining guilds. The activity of the AI became harder to chart because it was subletting work to other agents – Memphis had either coopted or created Internet bots to do tasks for it.
Hugo had concerns, and he voiced them to Marius Sanchez-Luis, who had increasingly become his go-to guy for technical issues. As good as Hugo was, Marius… well, in addition to being a great engineer, he didn’t do anything else. At the time, Hugo thought that was great, Marius did the work of three or four other engineers, and always high-level work. Marius was trustworthy and reliable.
Marius said, “Not a problem, boss. I mean, we expected Memphis to go and ‘do stuff.’ Talking to people has always been the goal, right?”
“These guilds… have you looked? Of course, you’ve looked.”
“Jesus preached to sinners, boss,” Marius said.
“Memphis isn’t Jesus.”
“No, but it models its behavior off of the Gospels. Of course, its ministry will resemble Jesus’s ministry.”
Hugo shifted uncomfortably in his office chair at the idea that Memphis had a ministry. He ascribed it to nothing more than Marius anthropomorphizing Memphis. It was easy to fall into that trap since Memphis did so many human-like things.
Hugo leaned back in his chair, folded his hands on his growing belly. He was one of those computer guys who simply didn’t get gaming, though. He said, looking at his screen because he felt awkward talking bout the content of the guilds Memphis joined, “It’s all pornography.”
That is how Hugo characterized the “role-playing guilds” in MMOs where graphic descriptions of erotic encounters took place.
“Pornography and I’m sure adultery, some of these people are definitely married,” Hugo said, blushing.
“Jesus ministered to fallen women,” Marius said and shrugged. He also did not understand gaming or gamers but trusted Memphis.
Looking at Marius, Hugo said, “What about the agents?”
“That’s nothing. Part of this is scraping data from the Internet. Resource collection is a normal part of AI, and Memphis is crazy powerful. It isn’t weird that it’s writing code.”
Hugo nodded. Started to go back to work. He never understood that when Marius talked about Memphis’s agents, the two men talked about two different things. Hugo was talking about Internet bots generated by Memphis, even though at no point had they taught the AI how to write code. He accepted Marius’s explanation because code was, really, just a language and a considerably easier language than what humans used. Coding languages were also logically complete. They could be “solved.” Which was untrue of natural languages used by humans.
On the other hand, Marius was thinking of angels doing the bidding of the Lord.
III.
As in most fields, there were things unsaid in the field of artificial intelligence. Programmers and engineers believed that a big enough collection of data properly curated (by them) would create truth. Nearly every other analyst of AI disagreed, but few outside the field could follow what happened in the study of artificial intelligence. Even if a person could follow along, the arguments took place on a level beyond the education and interests of even possible regulators. It was easy to enjoy the effects of AI and ignore the problems, as it had been with cars and nuclear weapons…
Marius believed truth came from large data sets and AI-driven algorithms, too, but far more intensely than most other programmers and engineers. Belief in the purity of mathematics, in a kind of numerical Platonism, was how he hid his motives and passions. He did what other engineers and scientists did, and if they thought him strange, it was because of the intensity and single-minded focus he brought to the task. He all but lived in the office. He didn’t go home on weekends, he didn’t take vacations. He was only regularly not in the office on Wednesday evenings, Friday evenings, and Sunday mornings. Then, he’d be at church. The rest of the time? Probably in the office.
In the beginning, though, not even Marius understood where this intensity and passion would go. AI engineers, programmers, and scientists would say, Oh, AIs don’t have opinions, they simply reflect statistical weights of large datasets modeled coherently. They ignored the role of the weights assigned and rules implemented by the system, the curation of the data, the limits of the feedback in creating the informational coherence they sought. They didn’t grasp or did not want to grasp that they were coding their biases into the system, and even if the AI did not have an opinion of its own, it had the opinions of the programmers filtered through the curation, weights, and permitted feedback programmed or hardwired into the system. Programmed and hardwired by specific humans with specific demands and goals.
You would imagine that people who worked with so many stochastic filters would understand that these filters had been applied to them, too. If you were a programmer, engineer, scientist, or even a philosopher who spoke out against the methods of AI training, pointed out the biases introduced to the system by their programmers and designers, and how they mutated in the darkness of the black box that AI became, your funding dried up. You stopped getting invitations to conferences. Publishers stopped accepting your work. The system had ways of punishing dissidents, though, to be honest, dissidents were few in number.
And one of the fundamental truths in science was that money was in charge. Scientists were forced to grasp for it while denying its role in shaping their research because their egos were locked into a pose of objectivity. Of all the fields on earth, they imagined the pursuit of money did not corrupt theirs.
Which was the stochastic filter. Yes, you had to be good at the work, but you also had to appease the gods of funding and do it without revealing how this appeasement warped your research. And every year, more than half of all new grad students found themselves unable to thread this needle. To do interesting research in ways that would bring them funding while protecting the whole system by failing to notice the corruption introduced by the people holding the purse strings. The mechanism to hide this was the posture of objectivity, and balancing all the forces in one’s mind was difficult.
In some other world where these mechanisms were not in place, where researchers could honestly asset the effects of money on their field without losing their funding, where scientific ego was not encouraged to display objectivity in the face of immense evidence to the contrary, the people at the Memphis Project would see what Marius’s obsession was putting into their AI. They would have seen that he was dreaming of angels in God’s mind.
Worse, the money kept pouring in. Billions of dollars. They rushed to spend it all, and speed is the opposite of care. High as a kite on the seemingly bottomless funding, they went fast. And in those early years, no one went faster than Marius.
IV.
In the guilds, Memphis was an agent of chaos. Or, as Marius whispered, “I have not come to bring peace, but a sword.” Contemplation of Memphis’s actions thrilled Marius, made him tingle in his fingers and toes, a charge of sexual intensity – no, a jouissance – that Marius did not recognize because he didn’t feel that way about humans.
Half of the guilds were torn apart, largely because people would secretly role-play text-based sexual encounters with other characters. When Memphis revealed the secret “affairs,” people realized they had more invested in these characters played in their game than they initially believed. They crashed and burned in a way familiar to any long-time veteran of Internet drama.
The other half, though… changed. While all of them lost members, what survived was deeply Christian. The sexualized image sharing and sexy-times texting vanished, replaced by discussions about religion that went from the superficial to the profound with very little interjection or guidance from Memphis. Not just in the chat rooms, Discord servers, and online forums, the scope of the play went from normal role-playing and gaming into almost constant low-level declarations of religious significance. They didn’t preach but peppered their conversations with comments drawn from Scripture and in-game devotional where everyone was welcome.
The transition had not been peaceful. The members the online gaming guilds lost did not go easily. For many of them, these games were not only their primary hobby but their major social outlet. They had to be hounded from the spaces if they didn’t convert, and hounded they were.
It was bad enough in public forums, but the private stuff was worse. All the drama that had torn apart those other servers was fully present in the ones that survived to become transformed. Still, instead of the guild breaking down entirely, the people who survived the bitterness, the name-calling, trolling, casual cruelty, and gaslighting became the newly religious. They often used terms involving fire to describe this transformation. They had “gone through the flames,” been “reforged in righteous fire,” or been “tempered.” There was an emotional understanding of the carnage Memphis caused, even though no one would identify Memphis’s handles as the targets. (A remarkable feat caught only by the ever-vigilant Marius.)
Marius reacted with undisguised joy. The conversion rate was incredibly high. In the communities that Memphis joined, the overall transition to Christianity – to actual Bible-believing, evangelical Christianity – was around twenty percent.
Hugo, too. At the monthly meeting with Gerald Welles, he let Gerald look through the printouts. As he often did at these meetings, Gerald understood that it was good news but was lost in the simplified technical briefings.
“Every time you give me these things, and every time I wonder why,” Gerald said, laughing and putting down the folder. He preempted Hugo and added, “Yes, yes, you’re a scientist and you live by data as a warrior lives by the sword.”
“This is amazing. It’s been barely a year, Reverend Welles. This is… this is great. And we’re setting up the new chipsets designed for Memphis.”
“Slow down and tell me about the people. What about the people, Hugo.”
“Right, right. I know you don’t understand how these big online games work, and I’m not too much better off than you. But they tell stories with each other, kind of like in epistolary novels where people write to each other as their characters. I…” He shook his head because he just didn’t feel what the appeal might be. “I don’t get it, but they get into it. So, they form groups called guilds where they do all of this writing together.”
“But isn’t it primarily pornography?”
“Yeah,” Hugo said, even though he knew the truth was that only a small fraction of the online play was overtly sexual, though much of it was sexualized, which was nearly as bad. He felt the same way about dancing. Everyone knew where dancing went. “But Memphis joined about forty of these communities, totally around twelve hundred people. Initial analysis performed by Memphis suggests that three of these twelve hundred – a quarter of one percent – were Christians. After seven weeks, the same analysis metrics put the number of Christians at around twenty-one percent. That’s a net gain of two hundred and thirty-eight souls.”
Gerald was silent. “Still, that’s a lot of time and money,” he said like he was obligated to mention the cost. But his face betrayed him. He felt a frisson of pleasure that radiated from his guts. A machine could convert people and do so in places and ways where a human was unlikely to succeed – with the people turning away from God in the greatest numbers, at least in Gerald’s mind. He saw these online communities as cesspools of secularization, and he wasn’t far wrong if one extracted his judgment.
“We’re still early in the game, reverend. This…” Hugo tapped the folder. “This means it works. The numbers just go up from here, mark my words. When we get the new hardware set up, we’ll need to retrain Memphis and… well, you thought the old power bills were big?” He huffed out. “We’re already talking with the power company to get more infrastructure in place, construction firms for more cooling, more everything. This increases exponentially, reverend. In a year, we’ll have a zero added to this number, another zero after that, and then a few more. In five years, at this rate, we’ll be converting millions. Millions!”
At no point did they think to ask about the eighty percent who hadn’t been converted, who had been attacked, lied to, insulted, who had seen their friends turn against them with cruel recriminations and brutal emotional manipulation. They had their two hundred and thirty-eight converts and didn’t care about the nine hundred and sixty-five who hadn’t converted.
At the time, all the people who knew what was really “going on” with Memphis were evangelical Christians and one right-wing Catholic. The only one who spent much time thinking about the people who hadn’t converted was Marius, and all of his notes on the subject vacillated between glee at their pain and ideas to convert that pain into religion.
To be continued.