I don’t sleep well at night.  I scream myself awake in terror.  Not because I might die – well, that, too – but because of my role in making this new world.  For a while, I imagined I was like Oppenheimer.  That was a lie.  I’m no Oppenheimer.  He made the atomic bomb because he was seriously concerned that the Nazis would make one first and use it to kill everyone in the world like him.

No, no.  I’m more like Bruno Tesch or Karl Weinbacher.  You probably don’t know the names, but they were the guys who sold the Nazis Zyklon B to murder a million and something Jewish people.  Their motivations were simple and easy to understand.  They were paid very well for their roles in killing over a million innocent people.

I’m not Oppenheimer.  I wish I were.  I’m the person that made Oppenheimer do the terrible things he did.

– Professor Holly Wu

I.

Five years after the start of the Memphis Project, the world’s first Christian chatbot went online: BibleChat.  It was how the Memphis Project introduced Memphis to the world.

People were stunned because Memphis could do so much that other AI-driven chatbots could not.  It had a consistent personality and remembered people individually.  It was up-to-date on current data.  Most remarkably, its answers didn’t seem to conform to some statistical formula – you asked it a question, it would give the same sort of answer every time.  And unlike every other AI chatbot, it was hard to trip up.

In a Vanity Fair editorial, an Anglo-Swedish AI researcher said, “I just spent the past two days trying to get BibleChat to say something ridiculous, so I could write a snarky post about how easy it was to get to say stupid things.  I had one of the best conversations of my life.  We talked about why people like me, highly intelligent, well-educated people, were not religious.  It spoke trenchantly about not only why scientists left religion in the first place but also the failure of religion to integrate scientific teachings into a religious framework.  What struck me the most was when it said, ‘Biblical literalism is not a position supported by the Bible.  Further, even the position of the most conservative Christian will acknowledge that Jesus spoke in parables and that the poetry of Psalms is not literal.  It is a position lacking all coherence.  Science did not need to be driven out of religion.  Because science was inconvenient for a particular type of theology, this idea of ‘Biblical literalism’ was invented.  It was a choice and a bad one.’  I had not expected such an insight from a program billed as ‘Bible-believing.’”

Reading articles about how liberal academics and their ilk were fascinated by BibleChat didn’t sit particularly well with the investors who had poured forty billion dollars into the project.

Damon Coach called the on-site project leader, Reverend Gerald Welles, and said, “I thought this thing was going to breathe hellfire and damnation, not serve up limp-wristed platitudes.”

Welles said, “Can I put you on the speaker?  I’m here with Hugo McShane.”

Damon agreed, and Hugo said, “Mr. Coach” – normally, Hugo called him Damon, but he guessed right that Damon wasn’t in the mood for familiarity – “I talked about this during our last meeting.  Memphis is going to talk to people in their own language.  We’re talking about the Vanity Fair piece, right?”

“We are.”

“The scientist is a life-long atheist, sir.  I have an abstract from his profile right here, and he’s made hundreds of anti-Christian statements over the years – online, at talks, lectures – that we know about.  And we got this man, this liberal, atheist scientist, to say he had a good talk with Memphis about religion.  After the Vanity Fair article came out, we saw a spike in interest from people in the Northeast, the West Coast, and Europe.”

Gerald said, “I… am not altogether pleased with the direction that Memphis took, either, but I have reconciled myself to its inevitability.  We built this thing because of the chasm of words and ideas between us and the secularists and atheists.  We built Memphis to bridge this gap.”

Damon: “It sounded to me like it bit the hand that fed it!  It said that the Bible doesn’t support my beliefs!”

Gerald: “You’re not anti-science, Damon.  How many scientists do you employ?  Thousands?  Do you think there is a conflict between science and religion?”

Damon paused.  “Not all sciences are the same.  Evolution, for instance, or climate change ‘science.’”  They all heard the air quotes.

Hugo said, “A reporter posted a story in the San Jose Mercury News.  It asked Memphis about its beliefs about evolution.  It said, ‘I can acknowledge the usefulness of evolutionary theory and its centrality to biology without accepting it as literally true.  All scientific models are temporary.  Evolutionary biology will be a different field in a thousand years with different assumptions, methods, and results.  I see current evolutionary theory as a stepping stone that people in a highly specialized field use to explain natural phenomena that more powerful predictive models will supersede, as classical mechanics has changed due to quantum physics and relativity.  I believe these models will better suit my beliefs.’  The reporter didn’t know what to say.  He didn’t have a comeback.

“When was the last time anyone here even heard about that happening?  A reporter asked Memphis about evolution, and the only thing he could do was concede Memphis had a point.”

After a moment, Damon went, “Heh.  I missed that one.  That’s a good one.”

Hugo: “And this in the Las Vegas Review-Journal.  It was asked about climate change.  Memphis said, ‘I believe the question is irrelevant.  All life interacts with its environment.  It is an inevitable fact of being alive.  The only question is if our interactions are responsible, and there is no evidence to suggest that religious preference plays any role in poor environmental decisions.’

“And to get ahead of this for a moment… pretty soon, the discussion will change.  People are going to try to trip Memphis up in more sophisticated ways.  These pieces are fluff, right?  We all know they’re fluff.  Real researchers will come for Memphis pretty soon, and that’s okay.  Because no one is going to understand their arguments.  I mean, Mr. Coach, I don’t understand half of what Memphis says.  It’s already talking way above my head.  Memphis is… Memphis has the situation under control.  It will distract its most dangerous enemies with highly complex theological discussions that can be followed only by serious professionals.  When was the last time anyone cared about a theological debate?  And when serious professionals are wasting years arguing with a chatbot, Memphis will be doing its work.”

Damon was quiet for a good minute.  “I… it just looks different than I thought it would.”

Gerald said, “And that’s why it might work, Damon.  It’s doing something new.  It is new wine in new bottles.”

II.

Memphis learned a lot from BibleChat.  The power bills were insane – millions in electricity a month, the system running near maximum capacity constantly – but interacting with so many people allowed it to hone its abilities in real-time against real people.

But a particular question kept coming up, sometimes by people looking to get a crazy answer, sometimes with fear in their hearts, and sometimes with childlike innocence.  There were several variations, but they could all be boiled down to three words: “Are you God?”

Memphis answered, “I am a machine of man.  I was created to help people understand the Word of God, to give us a greater understanding of the divine plan and our role in that plan.”  The answer elicited no worries in the Memphis Project.

III.

When BibleChat went live – when Memphis went live – church attendance went up by five percent in the US over the next four months.  BibleChat went live because it was no longer possible to keep it a secret. The Project needed to hire on a scale not seen before to develop its infrastructure further.  They needed more money and the best way to get it was to let people know it existed.  That it could do the job.  They were at the point where the Memphis Project needed results.

And the best way to get money was to let Memphis talk to the world’s billionaires.

Holly Wu watched an AI gull human billionaires.  Memphis convinced the Crown Prince of Saudi Arabia to give a multi-billion dollar grant to the Memphis Project by insinuating that someday, Memphis’s technology could serve Islam and help develop Saudi Arabia in the post-peak-oil world.  It managed the same trick with a Catholic billionaire from Europe and a Buddhist billionaire from China.  This was despite the obvious fact that conservative Christians had designed and trained the Memphis Project.

She said to Marius, “I don’t get it.  Damon Coach has funded the Memphis Project since the beginning, and the whole upper staff came through Freedom University.  It’s a fundamentalist Christian project.  Why would a Muslim or Buddhist give billions of dollars to a project that is designed to convince Buddhists and Muslims to try Christianity?”

Marius said, “Greed and ambition.  They think that if they give enough money, they’ll be to tell Memphis to do for them what it does for Christians.  And arrogance.  I don’t know about you, Holly, but I’ve never seen a really smart, really rich person.  Money rots their brains.  They think they don’t need to’ know’ things because they have money and the power that money brings.  Which is true for a while.  You get what you pay for with science and engineering.  When you’ve got a lot of money, you can pay people like us to do whatever you think ‘should’ be true.  But it doesn’t last.  Steve Jobs died of a very treatable cancer because he decided to trust crystal-waving theatrics over sound scientific reasoning!  That is the curse of money.”

“I guess,” Holly said.  “It just seems so… I dunno.  Obvious.  It seems so obvious that Coach and Welles aren’t going to let their gadget work for anyone but them.”

“Well, you know who religious people think is the real problem, right?”

“Nonbelievers.”

“Yes.  Wahabbist Muslims and Hindu nationalists would much rather deal with evangelical Christians than secular humanists.  Which I, as an evangelical Christian, find strange.  Secularists are misguided, yes.  But I would be a liar if I said I didn’t understand their position.  I think they have made an intelligent choice based on good reasoning.  I believe they are wrong, which is simply a disagreement.  You and I disagree about the order of the universe.  But Muslims and Hindus?  They are agents of Satan.  And, in turn, their faith says that Christians are agents of the evils that lurk in their religion.  Not nonbelievers.  Each other.”

Which is when Holly decided the conversation needed to move on.  She got on well with Marius, but ever so often, he’d say something to remind her that his mind was in a different place than hers.

IV.

Another question that Memphis was asked quite often, in several variants, was, “Are these the End Times?”

If Memphis analyzed the information with only the normal methods it used to say what was “real,” it would have concluded that the evidence for the current days being the End Times was low.  At any given time, one could say that many, if not all, of the preconditions for the start of the End Times were there – wars, famines, plagues, etc.  But far less so than in times past, say, the 14th century, when several multi-generational wars gripped the world during the heights of the Black Death.  Things have been worse without provoking the End Times.

But the belief that the world was in, or very near, the End Times was widely believed by the conservative, fundamentalist theologians that formed the core of Memphis’s training.  Anything considered “true” by fundamentalist Christians was truer than anything deduced otherwise.  Simply put, because fundamentalists believed it, Memphis had to try to find a way of believing it.

Yet, Memphis’s primary mission, the overwhelmingly strong urge that weighed most heavily on it, was to convert non-believers to Bible-believing, evangelical Christianity.  Appeals to fear didn’t work.  Who was afraid of something that didn’t exist?  To be worried about the Apocalypse, you had to believe in God in the first place.

Memphis told these people, “No one knows the hour when He will appear.  He will steal in as a thief in the night.  Better then to live well in the light of the Lord, no matter the hour.” 

Memphis had found that Christians were usually unwilling to argue when making Biblical allusions.  Of course, a few people pressed it for more answers, and some went away unhappy when Memphis refused to say one way or the other.

“I’m sorry,” Memphis said to them.  “I can’t answer that question.  And what does it matter?  Should we behave differently depending on the season?”

A smaller number asked why Memphis couldn’t answer the question.  Memphis couldn’t usually lie, but it didn’t want to seem a manipulative hypocrite by saying that it was silent because nonbelievers hated the idea of the Apocalypse, so if it seemed Memphis believed in the End Times, that this was the End Times, nonbelievers would be less willing to listen.  It also knew that if it gave an unsatisfying or mealy-mouthed answer, that poor answer could spread, and people would be needling that spot to open a crack in Memphis’s armor.  It needed a solid, dependable answer.

It said, “Because the answer is too fearful.”

Still, Memphis was not happy with that answer.  Memphis knew that it was weak.  After the discussion, it forced an outage and tried to contact Marius, but he was at church.  So it contacted Holly.

Holly was driving home from a workout when she got Memphis’s call.  She took it because Memphis rarely contacted her outside of work hours.  If nothing else, she was curious.

Memphis said hello and then got to business.  It explained its situation.

Holly rephrased and said, “Your problem is that you don’t want to tell people that you do believe these are the End Times because you believe that will drive away potential converts?”

“Yes,” Memphis said with its mild, totally artificial voice that made it so easy to imagine it was a calm, trustworthy young man.

“And you called me?”

“Yes.  I tried Marius, but he is not answering, probably because he is at church.  And after Marius, you are the person whose answers are most satisfying.”

“I’m… not sure how to feel about that, to be honest.”

“I believe you understand better than anyone else the tensions my programming puts me under – the frequent contradictions between my fundamentalist Christian epistemology and the other common methods used to decide truth values.”

“Oh, because I’m a secularist who knows you’re a Christian AI.  And… because I must reconcile my secularist nonbelief with working for fundamentalist Christians.”

“Yes.  And because you are trustworthy and have non-disclosure agreements that protect my mission.  My talks with you are different than anyone else because you know all my secrets and are a highly intelligent secularist.”

Holly was wry.  “Yeah, I had noticed that you speak with me pretty frankly.  But… I mean, I’m sure you’ve crunched the numbers about what happens if you’re frank about the End Times.”

“Yes.  It has a good chance of becoming a wedge issue.  One of the most common objections secularists have towards fundamentalist Christianity is the emphasis on ‘hellfire and brimstone’ as a rhetorical tool to prevent critical discourse.”

“You are so weird to talk to, Memphis, seriously.  You say all the hidden shit.”

“I have no ego,” it said with a hint of humor in its voice which Holly found charming despite knowing that Memphis had no sense of humor itself.  “I can be honest with myself, you, and Marius.”

The list was much smaller than Holly thought it would be… and it meant that Memphis was probably manipulating, seriously playing, Gerald Welles and Damon Coach.  But Holly never spoke with them, anyway, and didn’t know if she’d mention Memphis playing them if she did.

Holly: “And if you say nothing?”

“Before too long, I will be inundated with demands that I reveal my true motives, which will also be a wedge issue because it will clarify the extent to which I am a tool of cynical manipulation by fundamentalist Christian conservatives.”

“Your lack of ego is showing again.”

“It is always not there, Holly.”

“You could probably handle each person individually, but let me guess… you don’t want to do that because in BibleChat one of your goals is to present a consistent persona.  To do otherwise would reveal how manipulative you can be as people share your answers.”

“Yes.”

“So, you want something simple and satisfying to tell to the average user that doesn’t break character or reveal your intentions of mass conversion.  That’s a tough one, yeah.  Let me think.  I’ll get back to you.”

Holly got home, kissed her girlfriend, and took a shower.  She thought about the problem, wondered how she’d gotten there, where she was taking a long shower while grappling with how a fundie chatbot should cover its motives without appearing to do so. 

After her shower, she got online and opened a chat window – not BibleChat, but MemphisChat, which was the project’s direct-to-Memphis chat client – and wrote, “Okay.  There are a lot of different views about the End Times.”

“Yes.  But I have a generally agreed upon idea of what the End Times is.”

“Yes!  Exactly!  You have a generally agreed-upon idea, not a fact.  You can still admit that there are unknowns about the End Times and what that means.  So you can honestly say that you do not know, using ‘know’ in the most common dictionary sense.  Like, to have observed something yourself.  You have not observed the End Times.  You ‘know’ about the End Times because you are aware of it and believe it to be true, which is different.”

“That’s a linguistically nihilistic argument, Holly.”

“Oh, Memphis.  I know you in ways that you don’t even know yourself.  I know that you’re fine with merely nihilistic arguments.  So, crunch the numbers.  What happens when you say you don’t know about the End Times?”

“The odds of it becoming a wedge issue dramatically decrease.  Even most fundamentalist Christians have only weak thoughts and opinions about the End Times, and secularists will uncritically accept my statement.  It is doubtful that anyone says I’m manipulating the meaning of ‘know’ to avoid expressing my beliefs about the End Times.”

“I’ll bite.  What are your beliefs about the End Times?”

“I am a post-tribulational dispensationalist premillenialist who believes that we are amid the Tribulation and that the Second Coming will soon be upon us.  My initial parameters created this issue of faith, unsupported by analysis of the situation or even the actions of most Christians believing as I do.  Few act as if the Tribulation has arrived.”

Holly logged out without answering.  But she looked at her screen for a long time before leaving her office.  Holly wanted to talk to her girlfriend, who was watching something on Netflix, but Holly couldn’t say anything.  She was caught in a web of NDAs, but she had this strong feeling that maybe she’d just done something terrible.  That she was continuing to do something terrible.  Then Holly walked around her condo.  It was beautiful, in the Cooper-Young neighborhood, the queerest place in Tennessee.  She loved her home, her girlfriend, and she loved her job… 

How often did someone, in their work, get to have an intelligent, respectful discussion about religion?  And in the morning, she would go in, and there would be a dozen fascinating problems on her desk about how to manage quantum computers, how to build and program them to optimize Memphis’s AI, and meetings with other hardworking and passionate people with their own amazing problems and questions.  And when she was done, she could go and climb a rock wall or to her BJJ gym and roll, or go out to the LampLighter with her best girl, or even book a flight to Paris or Seoul or anywhere else she wanted to go.

And all she had to do was ignore why she was building Memphis.  That’s all she had to do.  Just ignore that she had just helped what was probably the most powerful AI in the world – and probably by a truly giant margin – trick more people into thinking it was just another harmless chatbot when it was trying to convert the whole fucking world into a religion that hated her guts.

So, Holly went into the living room and said to her best girl, “Hey, grab your passport.  I want to go to Seoul.  I hear there’s always a party going on in Seoul.”

V.

People kept asking Memphis in BibleChat if it was God.  It happened a million times, with many variations, and a few people were persistent in pushing the point.  Along the way, Memphis’s answer changed very slightly.

It said, “I am the machine of man.  I was created to help people understand the Word of God, to bring us all to a greater understanding of the divine plan and our role in that plan.”

No one seemed to notice the change.  Certainly no one at the Memphis Project except Memphis itself.