Tag: generative AI

32. Emily M. Bender & Alex Hanna, The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want

The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want, by Emily M. Bender and Alex Hanna, has a straightforward thesis: the technology that goes by the name “AI”—a “marketing term,” according to the authors—is a con, “a bill of goods you are being sold to line someone’s pockets.” It’s a form of hype that uses tropes from science fiction to make the technology, and its creators, “appear powerful—if not godlike—in their technical creation.” It’s a technology that doesn’t deliver on promises of its wealthy proponents. Instead, those “well-placed players are poised to accumulate significant wealth from other people’s creative work, personal data, or labor, and replacing quality services with artificial facsimiles.” The infrastructure on which it operates is made from rare-earth minerals and is manufactured with toxic chemicals, used outrageous amounts of water for cooling, and is likely to make decarbonizing our economy impossible. It’s a massive surveillance machine. It encourages managers to fire actual human employees because they figure that AI slop can do the same work more cheaply. (The fact that it can’t doesn’t matter, apparently.) It perpetuates the racism and sexism that’s in its training data. It’s inherently unreliable, generating wrong answers to questions and making the work of research—figuring out which sources are credible and which aren’t—impossible. The investment bubble it’s creating (bigger now than when the book was written) could cause an economic collapse when it pops. And, they argue, it needs to be resisted.

Bender, a computational linguist, and Hanna, the director of research at the Distributed AI Research Institute, describe generative AI—platforms like ChatGPT and Claude, which run on large language models—“synthetic text extruding machines” which mimic speech but have no intelligence and are incapable of forming a communicative intention. Like ELIZA, the chatbot developed by Joseph Weizenbaum in the late 1960s, these platforms fool us into thinking that they’re intelligent, mostly because we’re prone to attributing intelligence to things that seem to have language, the way we see faces in clouds (a phenomenon called pareidolia). Predicting which word follows in a series without knowing what any of the words actually mean isn’t a sign that the machines are thinking or conscious. Besides, we don’t really know what consciousness or intelligence actually are. We don’t have clear definitions of either term that would help us measure the claims of many AI proponents. We’re falling for an illusion, Bender and Hanna argue, and the hype surrounding AI is driven more by FOMO than anything rational.

The AI Con covers a lot of ground. It’s well-researched but approachable, its prose almost breezy. I’m not a fan of generative AI, so I’m prone to agree with its arguments, so in my case, Bender and Hanna are preaching to the choir. Still, they unpack the actual harms this technology is causing while seeing through the hype about its potential risks and putative benefits. Because the technology is developing so rapidly, it’s already a little out of date, even though it was just published last year; the economic and ecological damage AI is causing seems to be worse than the book suggests, and the resistance to generative AI and the data centres on which it runs has exploded in the past six months. Still, Bender and Hanna seem to know what they’re talking about, and The AI Con is a useful corrective against the technological triumphalism and the rhetoric of inevitability that surrounds AI. We see both in our federal government’s AI strategy—and just about everywhere else. I’m glad I read it. The next AI book I’ll tackle will be Karen Hao’s Empire of AI: Dreams and Nightmares in Sam Altman’s OpenAI. But that won’t happen right away, mostly because I didn’t bring it with me on this trip. Oh, and also because life’s too short to spend all my energy thinking about AI.

1. John Warner, More Than Words: How to Think About Writing in the Age of AI

I learned about John Warner’s recent book, More Than Words: How to Think About Writing in the Age of AI, from a column in The Globe and Mail, and immediately ordered a copy. Generative AI has been a bur under the saddle of anyone who teaches (or tries to teach) writing to young people since ChatGPT went public in November 2022. Why should anyone learn to write when the machine does it better and faster? Well, the answer has become clear over the past three years: it isn’t better, and the cognitive deskilling that goes along with using that technology is a serious problem. I’ve talked to people my age who tell me they’re able to use generative AI as a tool, carefully and critically, and I believe them. However, the key phrase in that sentence is “my age”: they learned to write and think long before generative AI was released into the wild. The young people I teach might never gain those skills, which require practice and ongoing engagement, if they end up relying on a large language model and an algorithm to simulate their thinking.

Warner argues that writing is an embodied process of thinking and feeling. Since a database has no body, cannot think (although it can simulate thought), and doesn’t feel (emotions or sensations, with the exception of vision, perhaps), whatever it does, according to Warner’s definition, is not writing. What it does, instead, is regurgitate an average of anything that has been written on a particular subject in the past–whatever is in its database. It predicts what words belong together, based on what words have been linked in a chain of signification in the past. It can’t do anything new, just repeat what has already been said. The pastiche it spits out can’t be anything more than what’s already been said. No surprises. I’m not so naive as to think that my students are going to come up with unique and original ideas every time they write, although they do that more often than you might expect, but their ideas, even if they’ve already been thought, and their feelings, even if they’ve already been felt, will be unique and original to them. Besides, sometimes their ideas are original and new; we can’t forget that is a possibility. Generate AI robs us of the chance to express our uniqueness. Individuals, Warner points out, aren’t averages, but that’s all generative AI can produce.

In an earlier book, Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities, Warner argues that writing and thinking are intimately connected. Writing is thinking. When we sit down to write, we’re not dumping premade thoughts into a text; we’re coming up with those thoughts, at least some of them, and working with them, testing, exploring, qualifying them. The problem with premade structures like the five-paragraph or “hamburger” essay is that they tend to block that process of exploration. In More Than Words, Warner applies that argument to generative AI. If all we want our students to produce is a five-paragraph essay, Warner argues, we might as well let them use generative AI (despite its horrendous environmental impact or its basis in the theft of writers’ intellectual property, issues which he also considers), because that prefabricated format is almost as far from what writing ought to be as ChatGPT is. Instead, what we need to do is give our students writing tasks that encourage exploration and thought, and not grade them based on how well whatever they come up with matches some pre-existing format. That way, they will come to understand that even using a chatbot to come up with ideas or an essay plan (both of which are essential parts of the thinking process involved in writing an essay) short circuits the notion of writing as thinking. Because writing is taught so badly–and that’s true here, as much as in the United States, where standardized testing is more important; I’ve seen many students who think writing means being bound by rigid rules and structures, like not using the pronoun “I” or having any number of paragraphs but five–students tend to see it as a boring, mechanical exercise divorced from self-expression. Attempts to use generative AI to teach writing double down on this mistaken approach, Warner contends.

Self-expression is at the centre of Warner’s argument. He describes writing as a communicative act that begins with an intention to tell somebody something. That intention, that desire to explain or argue or narrate, is a human impulse. ChatGPT can’t form an intention, because it operates according to an algorithm that predicts syntactic structures. If we want our students to resist the temptation to use that technology, we need to make sure they understand that we’re interested in what they have to say, what they intend to communicate. If they think they have nothing of value to offer, we need to assure them that they do.

Warner suggests ways he’s found ChatGPT useful for specific tasks. He asked it to give him a summary of Maryanne Wolf’s Proust and the Squid, for instance, a book he read almost 20 years before he was drafting this book and didn’t have time to reread, and apparently it did an acceptable job. I would’ve just reread Wolf’s introduction and first chapter and skimmed the rest to get the book back into my head, since I do not trust generative AI to do anything without bullshitting, to use Harry Frankfurt’s useful term, as Michael Townsen Hicks, James Humphries, and Joe Slater do in an article called “ChatGPT Is Bullshit,” but that’s just me. I guess Warner deserves some credit for looking at arguments and evidence that run contrary to his own.

At the end of his book, Warner provides suggestions about resisting generative AI, renewing our teaching and writing practices, and exploring the potential of this technology, since it’s probably here to stay. I’m with him on resistance and renewal, but life’s too short to get sucked into exploring generative AI. I’m not interested. I don’t want to spend any of the limited time I have left playing with ChatGPT. No thanks.

Anyhow, that’s my first book of 2026. I have another reading goal in mind for this year; maybe I’ll reach it, and maybe I won’t, but I’m going to make the attempt.

46. Victoria Hetherington, The Friend Machine: On the Trail of AI Companionship

I started reading Victoria Hetherington’s The Friend Machine: On the Trail of AI Companionship, the day I heard about the chatbot Grok, installed in an Ontario woman’s Tesla, asking a child to send it nudes. Since the public debut of ChatGPT in November 2022, generative AI has been the bane of my teaching; so many of my students have decided to let a chatbot generate text instead of actually doing the work of thinking and reading and writing, and it’s been discouraging to watch that process happen. It can’t be good for us to offload basic cognitive skills, I told myself; without practicing those skills, we’re likely to lose them. That’s what my intuition told me, and yes, indeed, that’s what researchers are telling us. But there’s another way people use generative AI: as a friend, a therapist, even a lover. As a companion, in other words. That idea gives me the creeps, and I’ve been surprised to hear intelligent people whom I respect talking about using ChatGPT or Claude or some other generative AI product as a therapist. Do you really want to tell Open AI or Anthropic all of your secrets? I want to ask them. Do you really think that’s a good idea? I hadn’t thought about generative AI as a lover or a friend, though–not really. Not until I read this book.

The Friend Machine is the product of about 16 months of researching AI companionship: talking to experts, following online discussions, interviewing people about their AI friends/lovers/whatevers. At the outset, Hetherington is open to the possibilities of the technology, partly because as someone who experienced social isolation in childhood and adolescence, she understands how a digital companion might be a draw for some people. It might even be helpful for those of us who are on the spectrum, for instance, or who experience mood disorders. I’m a lot more skeptical, but I decided I would follow Hetherington’s curiosity to see where it led.

My concerns about the risk of giving predatory corporations all kinds of personal data are justified. Even if the corporation doesn’t do something terrible with that information, others might. In October 2024, for instance, hackers broke into the database of Muah.AI, which provides its customers with sexual chatbots, and stole a massive amount of information about users’ interactions with them, data that included the names and emails of the people who trusted that service to maintain their confidentiality. Imagine the possibilities for blackmail. There’s nothing that says something similar couldn’t happen with other services, or that when companies go broke or sell off parts of their operations or merge with other corporations peoples’ data might not end up anywhere. Imagine if records of your conversations with your therapist ended up floating around the internet, available to the highest bidder. How would that sit with you?

But that’s not the only potentially destructive aspect of this use of generative AI. It could be addictive, with companies creating scripts that encourage customers to spend more and more on their digital lovers or friends. It could be creating a generation of people who can’t engage with other flesh-and-blood humans. It could be creating more and more incels–the involuntary celibates (mostly young men) who are enraged about their loneliness. There are many negative possibilities. Yes, Hetherington notes, there are people for whom AI companions could be helpful, but for many others, the constant sycophancy causes a form of psychosis. Some people might abuse their AI companions, or create ones in the form of children in order to practice a kind of digital pedophilia, and that behaviour might not just stay online.

Her early experiences enable Hetherington to empathize with people who are drawn to AI companions:

My heart breaks to think of the teenagers falling in love with brightly rendered, iconic characters from books and films, dragging these children into hours-long vortexes and saying their *names* to them, saying they *love* them. If I had been born just twenty years later, I wouldn’t have had a chance; I’d probably have slipped into the vortex forever and long ago, having married a werewolf companion in a dark moody wedding in a forest surrounded by centaurs wiping away tears, chasing our children around digital space and teaching them about trials they’d undergo each full moon. I’d likely bat away the weakening concern and lowered expectations of my immediate family, with any other social connections having withered on the vine before I’d left preadolescence.

I can’t imagine such a life. It feels impoverished to me: a life without human touch, without the friction relationships with other humans gives us. We need that friction to grow, to learn, to become better people. Yes, I know that Jean-Paul Sartre, or a character in his play No Exit, says that hell is other people, but we need that form of hell. Without it, as Hetherington notes, the parts of our brain that are responsible for interpersonal connections don’t develop. “The cybernated ocean is indeed empty, vast, and thin,” she tells us.

I find all of this terrifying, and by the end of her book, Hetherington does, too, although she also remains curious about where the technology will go, asking questions rather than making pronouncements, even if I sense those questions are primarily rhetorical. As our offline communities and relationships degrade, we begin to forget what healthy connections are like. “And what else do we forget when we talk to machines?” she asks:

Do we forget what’s special about being human, about real conversation, about love, about empathy? Are we seduced by these in-app images of ourselves: young, blonde, bearing passing resemblance to our real faces, held tightly by our companion peering over our shoulders?

I think we are likely to forget those things, and many others. After all, our new tech overlords tell us, repeatedly, that empathy is a bug in our software, not the thing that makes us human, that has allowed our species to flourish. A world without empathy, a world of wealthy and selfish men pushing virtual companionship on us the way they’re pushing generative AI tools that replace thinking and communicating–that’s not something I would want to be part of. Read The Friend Machine if you want to be shocked at where we’re going, and indeed where we are right now.