During one of my most desperate phases as a young novelist, I began to wonder if I should actually write my own stories. I was deeply indifferent at the time to anything resembling a plot, but I recognized that if I wanted to achieve any literary success, I had to tell a story that had a distinct beginning, middle, and end.
It was about twenty years ago. My grad school friends and I were obsessed with a website called the Postmodernism Generator that spit out absurd but hilarious critical theory articles. The site, which was created by a coder named Andrew C. Bulhak, who was building from Jamie Zawinski’s Dada Engine, is still up today and spawns scholarly forgery that reads: “In the works of Tarantino, a predominant concept is the distinction between creation and destruction. Marx’s essay on capitalist socialism argues that society has an objective value. But an abundance of appropriations concerning not theory, but sub-theory exist .
I figured, if a bit of code could spit out an academic paper, it could probably tell me what to write about. Most storylines I knew followed very simple rules, and since I couldn’t figure out how to line one up, I started talking to grad students in computer science about the possibilities of creating a bot that could say me who should go where, and what should happen to them. What I imagined was a simple text box where I could type a start – something like “A man and his dog arrive in a small town in Indiana” – and then the bot would just tell me that, on page 3, after six paragraphs of my beautiful descriptions and taut prose, the dog would find a mysterious set of bones in the backyard of their boarding house.
After a few months of research, it became clear to me that I wasn’t going to find much support for my plan. One of the computer science students, if I remember correctly, accused me of trying to take everything good, original and beautiful out of the creative process. The bots, he argued, could mimic basic handwriting and get better at it, but the AI could never tell you the way Karenin smiled, nor would it ever fixate on all the names of places that filled Proust’s childhood. I understood why he felt that way and I accepted to some extent. But I couldn’t see why a bot couldn’t just fill in all the parts where someone walks from point A to point B.
ChatGPT is the latest project released by OpenAI, a somewhat mysterious company from San Francisco that is also responsible for GIVE HIM, a program that generates art. The two caused a stir on social media, prompting people to share their creations and then immediately catastrophize over what AI technology means for the future. The chat version runs on GPT-3 – the abbreviation stands for “Generative Pre-Trained Transformer”, a pattern-recognition artificial intelligence that “learns” from huge caches of Internet text to generate believable responses to queries. The interface is refreshingly simple: you write questions and statements to ChatGPT, and it spits out remarkably consistently, even if every now and then hilariously wrong answers.
The concepts behind GPT-3 have been around for over half a century now. They derive from language models that assign probabilities to sequences of words. If, for example, the word “parsimonious” appears in a sentence, a language model will evaluate that word, along with all the words before it, and try to guess what should follow. Models require input: if your corpus of words only extends to, say, Jane Austen, then whatever your model produces will sound like a 19th-century British novel.
What OpenAI has done is power the internet through a language model; this then opened the possibilities for imitation. “If you adapt a language model to the Internet, you can regurgitate some really interesting patterns,” said Ben Recht, a friend of mine who is a professor of computer science at the University of California, Berkeley. “The internet itself is just patterns – much of what we do online is just knee jerk, like reactions to everything, which means most responses to things on the internet are quite predictable. So this just goes to show that.
GPT-3 itself has been around since 2020, and various people have already put it to the test. (The recent hype around it comes from the new version of chat.) In 2020, the Guardian asked the program to write an article about itself with a moderate, but not entirely disqualifying series of prompts from a human and reasonable, light editing. Gwern Branwen, writer and researcher, asked GPT-3 to write everything from poems to dad jokes. In one particularly illustrative example, Branwen fed the machine the opening to Shel Silverstein’s “Where the Sidewalk Ends” and asked it to fill in the rest.
This is the prompt – the first six lines of “Where the Sidewalk Ends”.
Here are the next six lines from Silverstein.
And here’s what GPT-3 came up with for what I felt was the next full stanza.
Thus, GPT-3 has trouble recognizing rhyme structure and is perhaps a little too indebted to “The Love Song of J. Alfred Prufrock” and its “women come and go / Talking of Michelangelo” lines. But it’s still remarkable that a computer can recognize the basic structure of a poem, seemingly figure out the tone of Silverstein’s verse, and then create what actually looks like a decent match to the original. (Though I’d say it reminds me a bit more of the opening pages of James Agee’s “A Death in the Family.”) The bot’s little words like “flax girls” are evocative, if a little superficially. The phrase “knows them by their faces” is actually quite nice.
The mind-blowing part was trying to recognize and analyze patterns in the bot’s responses. Was the line “people come and people go” really taken from TS Eliot, or is it just a series of random words that triggers the correlation in my head? So my response to the bot is not really a reflection of my relationship with technology, but rather my sense of my own knowledge. This raises a different question: why is my relationship to any other piece of text any different? To put it a little more specifically, why does it matter that a human or a bot tapped the wall of text?
#chatbot #rewrite