Bing’s Revenge and Google’s A.I. Face-Plant

This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors. Please review the episode audio before quoting from this transcript and email [email protected] with any questions.

Casey, we’ve seen two big AI announcements this week, one from Bing, and one from Google, called Bard. And I’m just curious. In this podcast, of the two of us, who would you say is more of a Bard and who is more of a Bing?

Hmm, well, I’ve drawn a great deal of inspiration, in my career, from the immortal bard, William Shakespeare. And so I would probably say the latter, for me.

I would also say Bard because you are frequently late and often underwhelming.

(LAUGHING) Oh, boy. Hey, watch out.

But I would say I am the Bing because I was laughed at for many years. And now no one’s laughing.

To be clear, I’m still laughing.

[THEME MUSIC]

I’m Kevin Roose. I’m a tech columnist at “The New York Times.”

And I’m Casey Newton from Platformer. This week on the show, OpenAI’s CEO, Sam Altman, and Microsoft CTO, Kevin Scott, on the new Bing, Google’s AI face plant, and then, nothing forever.

All right, so Casey, we had a field trip this week, our first joint “Hard Fork” field trip.

It was a true, whirlwind, 24 hours, up and back to Redmond Washington.

Yes, so we went to Redmond, just outside of Seattle. I got very excited on the way there because you got there a little earlier than I did. And you texted me that there was a Cheesecake Factory near our hotel. And I was very excited to go to the Cheesecake Factory.

And then you pulled a little switcheroo. And you said, actually, let’s go to this nicer restaurant, where they do not have 27-page menus. And I was a little sad.

Yeah, you couldn’t get both sushi and spaghetti there, unlike at The Cheesecake Factory. But we had a good time anyway.

We did.

And more importantly, we saw what I think later, looking back, we may see as maybe one of the more important days in tech, in 2023.

Yeah, so to back up, last week, a bunch of reporters, including both of us, get an email from Microsoft. And it’s a pretty cryptic email. It doesn’t have a lot of details in it. But it says, we’re going to be doing this announcement, at our campus in Redmond, Washington, on Tuesday. And you’re invited. But we’re not going to tell you what it is.

And this kind of thing, I think it’s safe to say, if I had gotten this email in 2018, I would have archived it because —

You would have said, I don’t want to hear about Microsoft Word version 36.

Right. I don’t want to hear about the new capabilities of the relaunched Excel. You and I have both been on a lot of these over the years. I’ve gotten pretty skeptical about this genre of show and tell because the details always leak beforehand. It’s never quite as exciting as you want it to be. You’re in a scrum of other reporters. It’s just not my favorite type of reporting trip.

And there’s also just so much hyperbole, right? It’s like, any company that wants to put on an event is going to tell you that everything is about to change. But the number of times that turns out to be true is generally pretty small.

Right. But this one, actually, I was very impressed by because what it turns out Microsoft was announcing is that they are relaunching Bing, their famously-mocked search engine, with Open AI’s AI built into it. So it’s basically — it’s an updated model of GPT — we think it’s GPT 4. Microsoft won’t say for sure — but built right into the Bing search experience.

Yeah, so two months ago, Chat GPT comes out. We started talking about it a lot. And one of the very first things that I think to myself is, when can I just do this in Google? And I think it’s fair to say, it was a real surprise that you were actually able to do it in Bing before you could do it in the most dominant search engine on the planet.

And I can’t, honestly, even imagine what went into re-architecting their entire search engine, so that it is powered by AI, and then also building that into their web browser, Edge, with a fairly decent set of 1.0 features. And it seems like they did that all within just a couple of months.

Yeah, it’s really impressive. And we should say that a lot of the new Bing features are similar to what’s in Chat GPT. You can have it write you an essay. You can talk to it like a therapist. You can even describe something you’re trying to code. And it will actually spit out the lines of code from what you type. But it feels really different to have this stuff built directly into a search engine.

Yeah, so the first thing to say is that Bing’s box, that you can search things in, is now just much bigger. You can enter up to 1,000 characters of text. And you can sign up for that wait list now. And eventually, you’ll be let in. And then you can essentially do anything on Bing that you have been doing on Chat GPT. Microsoft has said they are going to limit the number of queries that you can do. But they have not said what those limits are going to be. I assume people are going to push this thing to the absolute limit, just like they’ve been doing with chat GPT. So that’s the first thing.

The second thing is in this Microsoft Edge browser, which I think gets more interesting. So you install this browser. And right now, you have to use the developer version of the browser, which has the beta features. But basically, you surf the web normally. But there is now a Bing button in the top, right-hand corner. And if you open that, you can do a couple of different things.

One is, there is a Chat tab, which is, again, like chat GPT. Except you can now put up to 2,000 characters in there. So you could write —

A lot of characters.

— a lot of characters. You could ask at a very, very complicated set of questions. And then it will go ahead and spit back answers. It will remember the context of your conversation. So if you ask a follow-up question, then you don’t have to explain it all over again.

So this can become a pretty powerful research tool, particularly because — and I actually think this is huge — when the new Bing gives you an answer to your question, it adds a footnote. And you can click the footnote, and go to the web page where the information originally appeared.

Right. It cites its sources in a way that Chat GPT did not.

And we’ve talked about why that is a problem with Chat GPT. So much of the discussion is, OK, but is what this thing telling me — is it real? And there hasn’t really been any way to know. This thing is just predicting words and sentences.

But now, with Bing, you can actually see where it’s getting its information. That doesn’t mean you’re not going to have to hunt for it on the web page. It’s not perfectly sighted. But it’s way closer.

So that’s the first thing that you can do in this browser. The other thing is that they have this Compose feature. And now, this actually does get a little bit funny.

So you can tell the browser to write about whatever you like. And there’s a box where you can put something in. But you can specify the tone that you want it to write in. There are five available tones now, for human conversation. Those tones are professional, casual, enthusiastic, informational, or funny.

Which one are you going to use when you write emails to me?

Oh, it’s going to be casual, for sure.

Yeah.

Very short —

I’m going to write to you in formal, professional prose.

Dear Sir or Madam. And then you can also choose the format. So you can have it write a paragraph, an email, a blog post, or ideas. You can tell it how long you want it to write. And then you can generate a draft.

So think about all the people who are writing many of the same kinds of emails, over and over again. And they probably have some templates that they’re using. But now they can do this in a pretty automated way. It’s pretty easy to customize. And one of the demos that they did, on stage, at Microsoft this week, that just made me laugh, was they showed somebody using the tool to write a LinkedIn post.

It’s just like, if you’re a Microsoft, do you really want to be encouraging people to automate the creation of posts that were already half spam to begin with?

I’ve got to say, I felt a little scooped by that because I have also been using Chat GPT to write my LinkedIn posts.

Kevin, that hasn’t really been you, posting about your wins and your grind set?

No, it actually — I used this a couple of times to write LinkedIn posts. And it generated my most popular LinkedIn post ever.

And what was it about?

It was about AI. But it also ruined my life because it was like, it wrote me this LinkedIn post that solicited pitches for AI startups. (LAUGHING) And so for the next three weeks, I got buried in thousands of pitches, from tiny AI insurance startups.

Well, that serves you right for violating the trust of your LinkedIn followers.

Yeah, but it didn’t make me an influencer. So I’ll give it that. So we should just say what the search engine part of this looks like because in some ways, Bing now looks like kind of two products in one. So if you search for something on Bing, the left side of the screen is basically the same search experience we’re used to. It’s ads, it’s —

Links.

— blue links, it’s snippets. It’s the classic search experience. And then on the right side of the screen, for certain kinds of queries, it starts writing this AI-generated thing.

So for example, I searched, yesterday, just doing a demo, I searched, write me a menu plan for a vegetarian dinner party. And the left side of the screen popped up a bunch of recipes sites, and articles from food blogs about dinner plans for vegetarians. The right side of the screen actually did the GPT-style bullet-point list, where it gave me a menu of things that I could serve at my next vegetarian dinner party.

I then was able to click a little Chat button next to that, and ask a follow-up question, where I said, write a grocery list for this menu, sorted by aisle of the grocery store, including the amounts I need to make enough food for eight people. Which I was like, I don’t know if it’s going to get that. That’s a pretty hard query. And it did it.

Yeah, and this is complex stuff. This would have taken you a really long time to do. And I’m guessing everything that you just described took no longer than 60 seconds to type out and get the answers.

It was immediate. And I should say, it didn’t get everything right that I tried. So I asked it for a list of kid-friendly activities happening in Oakland this weekend because I was curious how it’s going to handle current events. And it gave me a list. But it was all things that had already happened. It was something that happened last weekend.

So it still needs some work. And I think Microsoft executives, who were there at the event, were very clear about the fact that this is not a perfect product, that they’re going to keep iterating and improving it over time. But it still does have some of these drawbacks, we’ve been talking about.

I think their basic feeling is, the stuff that it can do well is so cool that it makes up for the fact that, yes, it is broken in a lot of ways. So we’re very high on this technology. But what are the actual risks, here? What should we be watching out for, as this stuff actually enters into the mainstream?

So one obvious risk is just the risk of inaccurate information. That seems pretty glaring. And it’s going to be a while before it’s the main part of the search engine. Right now, it’s a sidebar. And I think it’s going to stay that way because it’s not super reliable yet.

Another thing I’ve been thinking about what this does to the whole industry of search-engine-optimized content because you have, right now, just tons and tons of websites that make all their money by gaming Google’s algorithms, by putting up recipes or gift guides —

What time is the Super Bowl.

What time is the Super Bowl — and that all risks being destroyed if people are no longer clicking the blue links on the left side of their search screen, if they’re just relying on the information that’s extracted from this training data, and put into this natural language response, on the right side.

So I think publishers are going to be really upset about this. I think there are going to be lawsuits. I think there’s going to be some attempt to get some more prominent attribution. As you mentioned, Bing, right now, gives you little annotations on some of the answers, that says, this answer came from Wikipedia, this answer came from the BBC, or whatever.

But they’re very small. They’re kind of hard to find. My guess is that not a lot of people are actually going to click through to the links. And so this is going to result in a big traffic drop for some publishers.

Yeah, and look, I’m a publisher. Presumably, my articles are going to get scraped and used to feed these things. And part of me feels like, well, yeah, that sucks. It would be cool if I could be compensated for that. But I don’t know if I feel that strongly enough that I don’t want tools like this to exist. Right? It’s just nice to be able to quickly have a robot do really accurate research on my behalf.

Yeah, I’ve been trying to replace you for years. So it’s very exciting.

Well, listen. We have a lot of questions. And so as part of our visit to Microsoft, we were able to put these questions to Microsoft CTO Kevin Scott and Sam Altman, the CEO of OpenAI. And we have that conversation coming up for you, right now.

[SOFT TECHNO MUSIC PLAYING]

Coming up after the break, Google puts the AI in fail.

That’s really good.

So we have this big, Microsoft event. We get on a plane we come back to San Francisco. And then on Wednesday, Google joins the party.

Yeah. And this was, I would say, the second time this week that Google tried to upstage Microsoft with AI-related announcements.

Right. So on Monday, Google announced its AI chat bot, its Chat GPT competitor, called Bard. And Bard is based on a version of LaMDA, which is Google’s language model. And Bard did not go so well in its first demo. In fact, in its first demo —

Wait, wait, wait, wait, wait. We actually can’t call it a demo because unlike Microsoft, unlike OpenAI, Google still has barely let anyone actually try this thing. All that Bard is, at this point, is a blog post. And so in order to have a successful pseudo launch of Bard, Google really only had one job. And that was to not screw up the one screenshot of Bard, answering a question. So that was the task ahead of them, Kevin. How did they do?

They did not do so well. So Google shared a sample question that was asked of Bard, in an animated GIF. And — do you say GIF or JIF?

I say GIF.

Yeah, GIF. This is a GIF podcast.

Yeah, it’s a GIF podcast. The G in GIF stands for Graphical.

Settled. So this GIF showed Bard, answering the question, what new discoveries, from the James Webb Space Telescope, can I tell my nine-year-old about? Bard offered three answers, including one that stated that the telescope took the very first pictures of a planet outside our solar system.

So pretty quickly, a bunch of astronomers on Twitter pointed out that this is wrong, that the first image of an exoplanet, which is what a planet outside of our solar system is called, was taken in 2004, not by the James Webb Space Telescope.

Now there’s subsequently been some debate and saying, well, maybe Bard was actually right because there is this one, specific exoplanet that the James Webb Space Telescope was the first telescope to take a picture of. So if you read it one way, it actually was right. And we made the mistake, as humans, in interpreting it.

But I think the way that most people interpreted that, is that Bard was wrong. So in its first demo — not even demo — in its first screenshot —

In it’s first screenshot — [LAUGHS]

Google’s AI chat bot made an error.

So lots to say about this. But when I read this headline, I immediately flashed back to a conversation I had with a former Googler recently. And we were talking about this Code Red inside the organization, to try to really galvanize energy around AI stuff. And I was asking this person how they thought it was going to go. And this person said, the thing about Google is it doesn’t do well when it is panicked.

And this person reminded me of the last time Google panicked, which was when Facebook was really encroaching on their territory. And they announced Google Plus. We all know how Google Plus went. And so look, I think it’s probably going to go better for Google, with AI. But oh, man, this is not the way to start.

Yeah, an inauspicious debut for sure. And then on Wednesday, Google held an event, which we did not attend in person, in part because it was in Paris.

Yeah. We would have loved to have attend.

We could be strolling down the banks of the Sien right now, enjoying like a nice cup of coffee and some macaron.

(IN FRENCH ACCENT) A croissant, a pain au chocolat and AI. Name a better morning. I dare you.

But we were not invited, and couldn’t have made it anyway because we were en route back from Redmond. But this event, which was live streamed from Paris, featured Google executives showing off some of Bard’s capabilities, and talking about AI improvements that they’re making to a number of other Google products, including Maps and Google Lens.

And while the Microsoft event, I think, generally raised investors’ hopes for Microsoft, this event, on Wednesday, by Google, seemed to have the opposite effect. In fact —

No, it literally had the opposite effect.

Yes. So after the event, shares in Alphabet, Google’s parent company, fell about 7 percent,

(LAUGHING) which is like, do you know how bad your tech demo has to go, for your stock to drop 7 percent? OK, this is probably a hiccup. I think it’s going to be fine. But it is really remarkable, in this moment, how not great this stuff is going for them.

Yeah, and it’s just such a stark contrast to last summer, if you can remember, when we were all talking about Google’s mysterious new AI technology, that an engineer left the company, got fired, because he claimed it was sentient. So we’ve gone from sentient, chat-bot mysteries to shares falling 7 percent because —

It can’t answer a telescope question.

— it can’t answer telescope question, in less than a year. And I just think that’s a remarkable turn of events.

It is. Also, I would say, looking through these announcements — and I should say, I have not had a chance to try any of these features. But none of this stuff is really making me super excited. The stuff we’ve been talking about today is stuff that will legitimately change work for millions, potentially billions of people. And the stuff Google announced this week is just not playing in the same league.

Yeah, it feels like incremental, small ball, some of it. And I will say, I think it’s a mistake to underestimate Google, here. They’ve got some of the best AI researchers and engineers in the world. They’ve got a huge, dominant lead in search, and other categories. And they have all the data and processing power they need.

So I think they’re going to get up this hill rather quickly. But it is just — I think one effect that Chat GPT has had on the tech industry, that I think is actually good, is that it has forced companies to put up or shut up. We are not going to be impressed by blog posts or research papers. Unless I can use a thing, it is not real to me.

You know what I read today, that I had forgotten? In 2016, Sundar Pichai stands up on stage, at Google I/O, and he says, as of today, Google is an AI-first company. That was almost seven years ago, that he said that.

And so look, there are trade offs to launching more quickly. We have a lot of concerns about what letting this stuff loose on the masses is going to mean for everyone. And yet, we also do think it’s cool. And we’re using it to do stuff. And that stuff is making our lives better and easier.

And it is just shocking that the company that invented the T and GPT is still this far behind. And actually, my question to you is, if you were Sundar Pichai, and you have what you have, do you do a blog post and an event this week, knowing what Microsoft is about to show off?

Interesting. I think I would probably wait. I think I would probably wait until my thing is ready for prime time, for the masses, because again, if it’s not — if I can’t log on and try it myself, it’s not real to me.

This is one thing that Apple has always really understood, is there’s no sense in previewing something. Even if you feel like all your competitors are six months or a year ahead of you, stick to your knitting. Finish the project. Say nothing about it until the moment when you’re ready to put it on sale, make it available for consumption. And if you do that, and the product is good, then you win.

Totally. What would you do if you were Sundar Pichai?

Here’s what I would do. I would say nothing until Bard was ready to be shown to the public. It’s one thing if they just want to, I don’t know, give a reporter a quote saying, yes, we do have something. You’ll hear about it soon. I think that’s fine.

But I don’t want to see another blog post, I don’t want there to be another event, until the stuff is ready for it to be shown, in exactly the same way that we saw this week, because at the end of the day, what users really care about is which of these services is better, which can I use now, if it’s going to cost me something, what is it going to cost me? Those are the relevant questions.

The relevant question is not, is Google working on something that will be available someday? We know that. So say nothing, and it’ll go better for you in the end. And I think it’s also just quite important that companies are doing all of the necessary safety work on these powerful language models, before throwing them open to the public.

I am really worried, as I know a lot of people in the AI industry are, about this arms-race quality that I asked Sam Altman about, where it is the case that Google is sprinting to get the stuff out the door, and whether it comes this week or next week or six months from now, my priority, as a human being, is for them to do the necessary work of making sure that these things are not going to be misused and weaponized, because I think that is a real danger, especially as these systems get better and better.

Yeah. All right, now let’s go look and see what that Space Telescope has been up to, since we’ve been talking.

If nothing else, I learned what an exoplanet is today.

[POP MUSIC PLAYING]

Coming up after the break, what is the deal with bias in AI?

All right, Kevin. So have you been watching this Twitch stream called, “Nothing Forever?”

I have not.

So it’s an art project from a couple of guys who had this idea, what if you could take generative AI, like the kind that powers everything we’ve been talking about today, and instead of using it to answer questions about telescopes and planets, what if you could make an episode of “Seinfeld” that just went on forever?

So it’s an never-ending, procedurally-generated episode of “Seinfeld,” where all of the lines are being written by AI, read by an AI voice, and there’s AI animation happening in the background?

That’s right.

So both the dialogue and the video are being created in real time by AI? There’s no human, animating this stuff?

Yeah. It’s not technically “Seinfeld” because that would be a copyright violation. And so instead of calling the character Jerry Seinfeld, the lead comedian in the show is called Larry Feinberg.

Everyone else has different names. Cosmo Kramer Zoltan Cackler.

No, come on.

Yeah. Anyway, it’s a very low-quality, pixelated scenes, bad, middy music, and the action just shifts from place to place, from Jerry slash Larry’s apartment, to all the other locations in the “Seinfeld” universe.

And the characters just talk to each other. There’s a laugh track. There are some attempts at jokes. It will cut to the comedy club where Larry is doing stand up. And people are just really struck by the weirdness of this. It is occasionally funny. It went viral because the characters appear to become self aware for a minute —

Oh, god.

— asking what they were all doing here. Actually, let me just play that for you.

[BAD DRUM MUSIC PLAYING]

(LAUGHING) OK. So the vibe is sort of “Simpsons” meets Minecraft. But it’s got some elements of Seinfeld. It cuts between exterior shots of an apartment building and a Minecraft-looking replica of the apartment from “Seinfeld.”

So clearly, it’s trained on some “Seinfeld” episodes. But it’s also — the dialogue is weird. The voices are all robotic. The graphics are very bad. So it’s not like — I wouldn’t watch this for more than a few minutes. But I do think it’s really interesting as art experiment.

Yeah, and we should say, there is a genius to this. So much of people’s TV consumption is just watching “The Office” from the start of season one until the very end, and then restarting it. If you could take the humor of that show, and generate endless riffs on it, in a never-ending episode of “The Office,” you could probably make a lot of money.

Someone just pitched that to Peacock.

Yeah.

So anyway, unfortunately, as does sometimes happen with these AI stories, this one does not have a happy ending. Recently, the Twitch stream was suspended after one of these comedy stand-up segments, when Larry Feinberg, the virtual comedian, said some things that were homophobic and transphobic.

No.

And they got —

Larry.

— yanked. Yeah. And it does seem like this is a place we’ve been before. It seems, like often is the case, that you will let an AI chat bot have a little bit of room to run. And next thing you know, that they are reflecting all of the biases in our society.

Do we know why it became transphobic and homophobic?

We do. And it’s super interesting. The dialogue in “Nothing Forever” was running off of GPT 3. And there are different flavors of GPT 3. The more sophisticated one, which “Nothing Forever” was using most of the time, is called Davinci. But it was causing some problems. And so the creators switched to a less-sophisticated model of GPT 3, known as Curie.

And the creators think that they accidentally stopped using Open AI’s content-moderation and safety tools when they switched over to Curie. So it was after they switched models, that the characters started saying the bad things.

That’s really interesting. How long had this thing been running before it broke bad?

For a couple of months.

A couple of months?

Yeah, it started up in December. And I think — I’ll speak only to the gay part of it, as a gay man. So one of the things that it said was, Larry Feinberg is musing about some comedy routines he might do in the future. And he proposes one that, quote, “all liberals are secretly gay and want to impose their will on everyone.”

And we should say, don’t be doing conspiracy theories about the gays. That’s bad. But that’s not the worst thing I’ve ever heard said about gay people. And I’m not saying that to let the AI off the hook. We do want these things to have controls. And we don’t want them to be running wild.

But I guess what I’m saying is, what I’m struck by is not, oh, my gosh, this AI just said the absolute most offensive thing that you’ve ever heard. I think what’s interesting is, it’s showing us how important it is to know what model you are using.

And here’s what I actually think that this story goes from a silly thing that happens online to something that has implications for anyone, working on this stuff. One of the stories I’ve been most fascinated by — we’ve talked about it on the show — is how CNET started to use this AI to generate articles, to provide SEO bait.

And one of my frustrations, in trying to understand the story, is CNET will not tell us which AI tools it is using. And so it becomes very difficult to evaluate how much we should trust this AI in the future because we literally don’t know what we’re using.

The way all of these tools work is, they’re trained on some body of material. And then they just predict answers based on that body material. Because that’s the case, it’s really important to know what the body of material is.

If you want to create an episode of “Seinfeld” that goes on in perpetuity, it would be really good to just train it on, I don’t know, every episode of “Seinfeld.”

Right.

Right? And maybe some extra supplemental material. And if you did that, you would probably have a pretty good chance of not saying something super transphobic or homophobic.

Right. And it’s also going to be a test for platforms that host user-generated content. and just to link this with the Microsoft visit, like the Microsoft executive who used Bing to write a LinkedIn post, there’s not that much different from doing that, than creating an endless “Seinfeld” episode.

Yeah.

We are entering a world where a lot of the content that gets submitted to social-media sites and posted online, that gets streamed on Twitch and YouTube, a lot of it is going to be algorithmically generated. And maybe there will be a human overseeing it and editing it, on the back end, before it goes up. But maybe there won’t. Maybe we’re headed into a world of never-ending “Seinfeld” episodes.

Well, you said to me yesterday, that you assume that, going forward, any email you receive will not have been written by a person.

Oh, yeah. I’m assuming that we have reached the singularity on this already. It’s going to become increasingly hard to tell whether the post you see on Instagram, or the email in your inbox, or even the video that you’re watching on Twitch or YouTube, was generated by a human or not.

And as Microsoft was demoing this new writing assistant, that it’s built into its web browser now, I was thinking, I may never get a PR email from a human again.

All right. So what do we do about all of this? I think that one of the answers, here, is just making people disclose what models they are using. And I’m going to say it. If you’re sending me email that was written by an AI, I actually want to know that. I don’t think I’m going to get that in Gmail. But I’d like it.

If I’m watching a stream, on Twitch, that’s generated by an AI, I do think it should disclose, this show is running off of this model. I want to be able to click on that model. I want to know where the training data came from. I want to know if it has moderation guidelines. We already have these community standards in place for all of these tech platforms. That’s been really good for their businesses. I think these large-language models should have similar kinds of disclosures because as you pointed out, this stuff really could damage trust for a lot of reasons. You never know who you’re talking to. We could rebuild that if we knew what technologies were baked into all these new tools we’re using.

Well, but to take the other side of that, when you watch a movie that has CGI graphics in it, they don’t tell you what program they used to create the CGI. We don’t expect that the things that we see on screen are all generated by human animators and directors. We accept that there’s some computer wizardry in there too. So how is that any different from getting an email from a PR firm, that was written by a robot?

Well, when Marvel uses a computer program to make Iron Man fly, that’s just a piece of art that is designed to stimulate and entertain me. When you are using it to write emails, the assumption that everyone is trading on is that a person wrote this. And if that’s not going to be true, then I just think that we should have some basic questions answered for us.

Yeah. Yeah, I can see there being — when your email program puts a little thing at the bottom, that says, sent from my iPhone, please excuse typos or whatever. I think we will start to see some programs that put little disclaimers in there, like this email was drafted by Bing, or this email was drafted by GPT.

Yeah, because what else is going to happen, is just in the same way that fake “Seinfeld” broke bad, these AIs, that people are using to write emails, are going to say stupid or offensive things, so that people are just scrambling, like, oh, it wasn’t me. It was the AI. And if you actually have a disclosure in there, you’ll have more plausible deniability.

Exactly. I can say all kinds of rude and offensive things over email now. And if people question me about it, I can just say it was Bing’s fault. Thanks, Bing. [MUSIC PLAYING]

All right, well, I’d love to end this on a “Seinfeld” joke. But I really haven’t watched it much over the past 20 years.

Before we go, I just want to thank our listeners. We’ve been getting so many great emails, including a number that urged us to check out this endless “Seinfeld” episode that we just discussed. So thank you to everyone who wrote in, suggesting that we take a look at that. Please keep sending us your ideas.

We really are listening to you. I also wanted to shout out our reader, Robin, who wrote, “in one of your latest episodes, you mentioned CNET’s false AI article on compound interest. I was struggling in my high school econ class, and stumbled across the article before it was corrected. As you can probably guess, I didn’t do so hot on my latest tests.

I feel kind of silly for not spotting the inaccuracies.”

Robin, don’t be so hard on yourself. You were misled by a rogue AI. But hopefully, you’ve now learned to be on alert.

Yeah, and please don’t use Bard for your next astronomy quiz.

[THEME MUSIC]

“Hard Fork” is produced by Davis Land. We were edited this week by Jen Poyant and Paula Szuchman. This episode was fact checked by Caitlin Love.

Today’s show was engineered by Alyssa Moxley. Original music by Dan Powell, Elisheba Ittoop, Marion Lozano, and Rowan Niemesto. Special thanks to Hanna Ingber, Neil Gallagher, Kate LoPresti, and Jeffrey Miranda. That’s all for this week. The solution for Wednesday’s Wordle was flail.