Bing’s Revenge and Google’s A.I. Face-Plant - The World News

Bing’s Revenge and Google’s A.I. Face-Plant

This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors. Please review the episode audio before quoting from this transcript and email [email protected] with any questions.

kevin roose

Casey, we’ve seen two big AI announcements this week, one from Bing, and one from Google, called Bard. And I’m just curious. In this podcast, of the two of us, who would you say is more of a Bard and who is more of a Bing?

casey newton

Hmm, well, I’ve drawn a great deal of inspiration, in my career, from the immortal bard, William Shakespeare. And so I would probably say the latter, for me.

kevin roose

I would also say Bard because you are frequently late and often underwhelming.

casey newton

(LAUGHING) Oh, boy. Hey, watch out.

kevin roose

But I would say I am the Bing because I was laughed at for many years. And now no one’s laughing.

casey newton

To be clear, I’m still laughing.

[THEME MUSIC]

kevin roose

I’m Kevin Roose. I’m a tech columnist at “The New York Times.”

casey newton

And I’m Casey Newton from Platformer. This week on the show, OpenAI’s CEO, Sam Altman, and Microsoft CTO, Kevin Scott, on the new Bing, Google’s AI face plant, and then, nothing forever.

kevin roose

All right, so Casey, we had a field trip this week, our first joint “Hard Fork” field trip.

casey newton

It was a true, whirlwind, 24 hours, up and back to Redmond Washington.

kevin roose

Yes, so we went to Redmond, just outside of Seattle. I got very excited on the way there because you got there a little earlier than I did. And you texted me that there was a Cheesecake Factory near our hotel. And I was very excited to go to the Cheesecake Factory.

And then you pulled a little switcheroo. And you said, actually, let’s go to this nicer restaurant, where they do not have 27-page menus. And I was a little sad.

casey newton

Yeah, you couldn’t get both sushi and spaghetti there, unlike at The Cheesecake Factory. But we had a good time anyway.

kevin roose

We did.

casey newton

And more importantly, we saw what I think later, looking back, we may see as maybe one of the more important days in tech, in 2023.

kevin roose

Yeah, so to back up, last week, a bunch of reporters, including both of us, get an email from Microsoft. And it’s a pretty cryptic email. It doesn’t have a lot of details in it. But it says, we’re going to be doing this announcement, at our campus in Redmond, Washington, on Tuesday. And you’re invited. But we’re not going to tell you what it is.

And this kind of thing, I think it’s safe to say, if I had gotten this email in 2018, I would have archived it because —

casey newton

You would have said, I don’t want to hear about Microsoft Word version 36.

kevin roose

Right. I don’t want to hear about the new capabilities of the relaunched Excel. You and I have both been on a lot of these over the years. I’ve gotten pretty skeptical about this genre of show and tell because the details always leak beforehand. It’s never quite as exciting as you want it to be. You’re in a scrum of other reporters. It’s just not my favorite type of reporting trip.

casey newton

And there’s also just so much hyperbole, right? It’s like, any company that wants to put on an event is going to tell you that everything is about to change. But the number of times that turns out to be true is generally pretty small.

kevin roose

Right. But this one, actually, I was very impressed by because what it turns out Microsoft was announcing is that they are relaunching Bing, their famously-mocked search engine, with Open AI’s AI built into it. So it’s basically — it’s an updated model of GPT — we think it’s GPT 4. Microsoft won’t say for sure — but built right into the Bing search experience.

casey newton

Yeah, so two months ago, Chat GPT comes out. We started talking about it a lot. And one of the very first things that I think to myself is, when can I just do this in Google? And I think it’s fair to say, it was a real surprise that you were actually able to do it in Bing before you could do it in the most dominant search engine on the planet.

And I can’t, honestly, even imagine what went into re-architecting their entire search engine, so that it is powered by AI, and then also building that into their web browser, Edge, with a fairly decent set of 1.0 features. And it seems like they did that all within just a couple of months.

kevin roose

Yeah, it’s really impressive. And we should say that a lot of the new Bing features are similar to what’s in Chat GPT. You can have it write you an essay. You can talk to it like a therapist. You can even describe something you’re trying to code. And it will actually spit out the lines of code from what you type. But it feels really different to have this stuff built directly into a search engine.

casey newton

Yeah, so the first thing to say is that Bing’s box, that you can search things in, is now just much bigger. You can enter up to 1,000 characters of text. And you can sign up for that wait list now. And eventually, you’ll be let in. And then you can essentially do anything on Bing that you have been doing on Chat GPT. Microsoft has said they are going to limit the number of queries that you can do. But they have not said what those limits are going to be. I assume people are going to push this thing to the absolute limit, just like they’ve been doing with chat GPT. So that’s the first thing.

The second thing is in this Microsoft Edge browser, which I think gets more interesting. So you install this browser. And right now, you have to use the developer version of the browser, which has the beta features. But basically, you surf the web normally. But there is now a Bing button in the top, right-hand corner. And if you open that, you can do a couple of different things.

One is, there is a Chat tab, which is, again, like chat GPT. Except you can now put up to 2,000 characters in there. So you could write —

kevin roose

A lot of characters.

casey newton

— a lot of characters. You could ask at a very, very complicated set of questions. And then it will go ahead and spit back answers. It will remember the context of your conversation. So if you ask a follow-up question, then you don’t have to explain it all over again.

So this can become a pretty powerful research tool, particularly because — and I actually think this is huge — when the new Bing gives you an answer to your question, it adds a footnote. And you can click the footnote, and go to the web page where the information originally appeared.

kevin roose

Right. It cites its sources in a way that Chat GPT did not.

casey newton

And we’ve talked about why that is a problem with Chat GPT. So much of the discussion is, OK, but is what this thing telling me — is it real? And there hasn’t really been any way to know. This thing is just predicting words and sentences.

But now, with Bing, you can actually see where it’s getting its information. That doesn’t mean you’re not going to have to hunt for it on the web page. It’s not perfectly sighted. But it’s way closer.

So that’s the first thing that you can do in this browser. The other thing is that they have this Compose feature. And now, this actually does get a little bit funny.

So you can tell the browser to write about whatever you like. And there’s a box where you can put something in. But you can specify the tone that you want it to write in. There are five available tones now, for human conversation. Those tones are professional, casual, enthusiastic, informational, or funny.

kevin roose

Which one are you going to use when you write emails to me?

casey newton

Oh, it’s going to be casual, for sure.

kevin roose

Yeah.

casey newton

Very short —

kevin roose

I’m going to write to you in formal, professional prose.

casey newton

Dear Sir or Madam. And then you can also choose the format. So you can have it write a paragraph, an email, a blog post, or ideas. You can tell it how long you want it to write. And then you can generate a draft.

So think about all the people who are writing many of the same kinds of emails, over and over again. And they probably have some templates that they’re using. But now they can do this in a pretty automated way. It’s pretty easy to customize. And one of the demos that they did, on stage, at Microsoft this week, that just made me laugh, was they showed somebody using the tool to write a LinkedIn post.

It’s just like, if you’re a Microsoft, do you really want to be encouraging people to automate the creation of posts that were already half spam to begin with?

kevin roose

I’ve got to say, I felt a little scooped by that because I have also been using Chat GPT to write my LinkedIn posts.

casey newton

Kevin, that hasn’t really been you, posting about your wins and your grind set?

kevin roose

No, it actually — I used this a couple of times to write LinkedIn posts. And it generated my most popular LinkedIn post ever.

casey newton

And what was it about?

kevin roose

It was about AI. But it also ruined my life because it was like, it wrote me this LinkedIn post that solicited pitches for AI startups. (LAUGHING) And so for the next three weeks, I got buried in thousands of pitches, from tiny AI insurance startups.

casey newton

Well, that serves you right for violating the trust of your LinkedIn followers.

kevin roose

Yeah, but it didn’t make me an influencer. So I’ll give it that. So we should just say what the search engine part of this looks like because in some ways, Bing now looks like kind of two products in one. So if you search for something on Bing, the left side of the screen is basically the same search experience we’re used to. It’s ads, it’s —

casey newton

Links.

kevin roose

— blue links, it’s snippets. It’s the classic search experience. And then on the right side of the screen, for certain kinds of queries, it starts writing this AI-generated thing.

So for example, I searched, yesterday, just doing a demo, I searched, write me a menu plan for a vegetarian dinner party. And the left side of the screen popped up a bunch of recipes sites, and articles from food blogs about dinner plans for vegetarians. The right side of the screen actually did the GPT-style bullet-point list, where it gave me a menu of things that I could serve at my next vegetarian dinner party.

I then was able to click a little Chat button next to that, and ask a follow-up question, where I said, write a grocery list for this menu, sorted by aisle of the grocery store, including the amounts I need to make enough food for eight people. Which I was like, I don’t know if it’s going to get that. That’s a pretty hard query. And it did it.

casey newton

Yeah, and this is complex stuff. This would have taken you a really long time to do. And I’m guessing everything that you just described took no longer than 60 seconds to type out and get the answers.

kevin roose

It was immediate. And I should say, it didn’t get everything right that I tried. So I asked it for a list of kid-friendly activities happening in Oakland this weekend because I was curious how it’s going to handle current events. And it gave me a list. But it was all things that had already happened. It was something that happened last weekend.

So it still needs some work. And I think Microsoft executives, who were there at the event, were very clear about the fact that this is not a perfect product, that they’re going to keep iterating and improving it over time. But it still does have some of these drawbacks, we’ve been talking about.

casey newton

I think their basic feeling is, the stuff that it can do well is so cool that it makes up for the fact that, yes, it is broken in a lot of ways. So we’re very high on this technology. But what are the actual risks, here? What should we be watching out for, as this stuff actually enters into the mainstream?

kevin roose

So one obvious risk is just the risk of inaccurate information. That seems pretty glaring. And it’s going to be a while before it’s the main part of the search engine. Right now, it’s a sidebar. And I think it’s going to stay that way because it’s not super reliable yet.

Another thing I’ve been thinking about what this does to the whole industry of search-engine-optimized content because you have, right now, just tons and tons of websites that make all their money by gaming Google’s algorithms, by putting up recipes or gift guides —

casey newton

What time is the Super Bowl.

kevin roose

What time is the Super Bowl — and that all risks being destroyed if people are no longer clicking the blue links on the left side of their search screen, if they’re just relying on the information that’s extracted from this training data, and put into this natural language response, on the right side.

So I think publishers are going to be really upset about this. I think there are going to be lawsuits. I think there’s going to be some attempt to get some more prominent attribution. As you mentioned, Bing, right now, gives you little annotations on some of the answers, that says, this answer came from Wikipedia, this answer came from the BBC, or whatever.

But they’re very small. They’re kind of hard to find. My guess is that not a lot of people are actually going to click through to the links. And so this is going to result in a big traffic drop for some publishers.

casey newton

Yeah, and look, I’m a publisher. Presumably, my articles are going to get scraped and used to feed these things. And part of me feels like, well, yeah, that sucks. It would be cool if I could be compensated for that. But I don’t know if I feel that strongly enough that I don’t want tools like this to exist. Right? It’s just nice to be able to quickly have a robot do really accurate research on my behalf.

kevin roose

Yeah, I’ve been trying to replace you for years. So it’s very exciting.

casey newton

Well, listen. We have a lot of questions. And so as part of our visit to Microsoft, we were able to put these questions to Microsoft CTO Kevin Scott and Sam Altman, the CEO of OpenAI. And we have that conversation coming up for you, right now.

[SOFT TECHNO MUSIC PLAYING]

casey newton

Well, thanks for sitting down with us. Really appreciate it. Very exciting demo.

Sam, when I saw the new Bing search-result page, with the blue links on the left side, and the AI-generated answer on the right side, my thought was, a, that’s very cool, b, publishers are going to lose their minds over this because why would I click a blue link, as a consumer, when I can get a natural-language answer, right there, on the other side? There’s a whole industry in search-engine optimization and how to appear higher in the results. So how are you both thinking about the reaction that this might get from the companies that depend on people clicking the blue links?

sam altman

I think Microsoft did a really nice job of putting links into that generated thing, on the right, as you see. One of the criticisms — a fair criticism of chatgpt is you’re not sure if it’s accurate or not. And so by adding all of the citations that Microsoft has done, I think that will drive traffic out.

And also, if, in the example of buying a TV, if the click that happens at the end of that is to the top TV review, and then you would go buy from them, and your intent and likely to purchase through that, or whatever, is much higher, I think the quality of referred traffic should get super high, even if there’s less volume.

casey newton

Hmm. And Kevin, what do you think?

kevin scott

Look, for everything that’s different about this, there are new opportunities for publishers and advertisers and users to use the technology in a different way. So I think, to Sam’s point, there’s going to be plenty of value there, plenty of distribution for publishers, plenty of ways for people to participate in what this new marketplace is.

casey newton

So I’m also curious about how the experience is going to be different in search versus what I’ve been doing in Chat GPT. With chat GPT, I think your training data stops in 2021. With search, people want results that are a lot fresher, might be about something that is happening right now. So like can you quickly update the models? Or is this stuff going to be on a bit of a lag?

kevin scott

No, no, no. So the way the technology actually works is, we use the model to go from the prompt to a set of queries that actually get executed on a live index. We pull that information back. And then we give it to the model again, to summarize, and to complete whatever it is that the user wanted, from the initial prompt. So it’s always up to date with the latest information.

casey newton

Sam, one thing we’ve seen, with Chat GPT, since it’s been out, is that it’s not infallible. It makes lots of mistakes, including math mistakes, just inventing things out of thin air. You’ve said that Chat GPT shouldn’t be used for anything important. How are you thinking about that, and plugging that into what could be a giant, popular search engine that’s used by tons of people every day? Are you worried about the mistakes that this AI could make?

sam altman

First of all, it’s a better model that makes less mistakes. But there’s still a lot of mistakes that come out of the base model. But as Microsoft went through today, and all of the layers they’ve put on top of that, and all the technology they’ve built around it, there will still be mistakes. But the accuracy and utility — check it out yourself. Don’t take my word for it. But I think it is significantly improved over the Chat GPT experience. And it will get rapidly better over time.

casey newton

Are there categories of search, or types of search, that people should be more cautious about doing on Bing, that may be more prone to errors than others?

sam altman

I don’t the answer to that yet. Obviously, there are whole categories of queries that we’re going to try to really moderate, just because you can imagine some of the harms that could come from them.

I think we’re going to learn, pretty quickly, where the actual mistakes are, that the model makes. And every time we launch something, we learn so much, just from launching it and seeing how users are interacting with it. We learned a ton in copilot. There’s an incredible amount of learning that’s come from Chat GPT. One thing I would point out is just the newness of all of this technology. Three years ago, GPT 3 hadn’t launched. I think very few people believed that meaningful AI progress was going to happen.

It’s really only been a little bit over two months that, I think, this has had mainstream attention and interest and trying to figure out how we’re going to use all this, how things will adapt. And with any new technology, you don’t perfectly forecast all of the issues and mitigations. But if you run a very tight feedback loop, at the rate things are evolving, I think we can get to very solid products, very fast.

casey newton

Presumably, you guys have been using this technology, or using the new Bing, for at least a few weeks, I would imagine. I’m curious. How has it changed the way you search? Or what are you using it to do, that maybe felt inaccessible with previous search tools?

sam altman

It very quickly becomes indispensable. I miss it when I don’t have access to it. And it’s everything from whimsical queries — I have a 14-year-old daughter who like says things like “riz,” and “bussing.” And I have no idea what she’s talking about. And you can type this in, like, hey, I’ve got a teenager. She’s saying these words. I don’t understand it.

casey newton

You have a Gen Z translator?

sam altman

Right.

casey newton

You automated the cool dad.

sam altman

And it’s really, really good at stuff like that. Just a nerdy thing that I did yesterday is, I’m a maker. And I was looking for industrial bartack’s sewing machines, which I have previously heavily researched. And using the new Bing, it showed me something that I’d never seen before, in any of my other searches. And it actually identified the machine that I think I’m going to go buy.

casey newton

Wow.

kevin scott

I’ll say it a different way, which is, I’ve always — given how close we are with Microsoft, I’ve always felt a little bit bad that I use Google and not Bing. But now I don’t think I’ll use Google anymore.

casey newton

Really?

sam altman

You just really don’t want to go back.

casey newton

What about Chrome? Are you going to switch over to Edge?

sam altman

So much of my workflow is built into — yeah, I would like to get there, but — look, I’m happy for people to use whatever technology makes sense to them. This isn’t an attempt to Jedi mind trick the whole world into using something that’s not the best possible choice for them. People will choose what’s doing useful things for them.

casey newton

Yeah. Just as a quick aside, once Chat GPT launched, I talked to the Google folks, who were clearly very jealous, and they would say to me, well, you have to understand, Chat GPT is this a demo. And the implication was, we’re building the real stuff back here. And you’ll see it someday.

But then you looked two months later, and nobody was treating Chat GPT like a demo. People were just absolutely using it as part of their everyday lives. And I imagine we’re going to see something similar with Bing.

sam altman

Yeah, Chat GPT is all horrible product.

And it was really not designed to be used in people’s —

casey newton

Wait a minute. How would you say that, though, Sam? Because it was a box you could type in, that you could get almost anything you wanted.

sam altman

For sure. But if you think about the way people want to integrate it into their workflow, the way that you want it to be able to use your own data, to integrate with your other applications and services, people are really just going to a site that sometimes works and sometimes it’s down. They’re typing in something. They’re trying to get until they get it right. And they’re copying the answer, and going to paste it somewhere else, and then going back, and trying to integrate that with search results. Or there are other workflows.

It is cool, for sure. And people really love it, which makes us very happy. But no one would say this was a great, well-integrated product yet.

casey newton

It’s not the 1.0 version.

sam altman

Yeah, it’s the 0.7 or something like that. But there’s so much value here, that people are willing to put up with it. And this is a real product now.

casey newton

Kevin, I’m thinking a lot today, as I’m hearing Microsoft executives talk about AI, about Tay, which was the infamous chat bot from 2016

sam altman

That’s a name I have not heard in a long, long time.

kevin scott

This chat bot, it came out. It was almost immediately seized on by trolls, who taught it awful things, and turned it into a racist, Holocaust-denying disaster. And it got pulled down.

And I think that became a cautionary tale for a lot of people in the AI industry. And I think actually, if you talk to people at these big AI companies, it’s a lot of the reason they haven’t launched stuff as quickly as they have, because they remember that as a moment of, oh, god, what have we done.

Was it hard to rally support internally, at Microsoft, for this? Were there people who said, wait a minute. Didn’t we learn our lesson about releasing an AI chat bot last time? Is there still scar tissue from that?

sam altman

Well, so I suspect there is scar tissue. The interesting thing about Tay is, that was, I think, one year before I started here, at Microsoft. So I don’t the full history.

One of the interesting things, though, that we learned from it wasn’t, oh, my god, we should never do this again. It’s like, oh, my god, we need to really get serious about responsible AI safety and ethics. And it was one of the catalyzing events because we were still very, very serious about AI, and had high conviction that it was going to be one of the most important, if not the most important technology, that any of us have ever seen, that we had to go solve all of these problems to avoid making similar mistakes again. Which was not to say that we won’t make mistakes. But we have spent the past 5 and 1/2 years trying to really — not just articulate a set of principles, so that we can push them out as marketing materials, but how do we put them into practice every day, in the work that we’re doing?

And so in that sense, when this new breakthrough came from OpenAI, we didn’t have as much of that resistance as you would imagine. They knew what muscle they were going to go exercise, to ensure that we could launch it with confidence, that we were being safe and responsible.

kevin scott

Yeah, it also probably helps that this technology is useful, whereas Tay was a novelty chat bot, I think.

sam altman

A lot of things in that era got launched by research teams, just because they were super excited about, hey, I’ve got this thing that technologically is interesting. And they weren’t really thinking about the product implications.

kevin scott

Right.

casey newton

Well, and on that front — I’m sure you saw the story this week — there’s a Reddit community that is now devoted to jail breaking Chat GPT. And they figure out a way to do it. And then you guys quickly patch it. And do you think that that’s just going to be an ongoing, cat-and-mouse dynamic forever? Or —

sam altman

No because I think where we are right now is not where we want to be. The way this should work is that there are extremely wide bounds of what these systems can do, that are decided by, not Microsoft or OpenAI, but society, governments, something like that, some version of that, people, direct democracy. We’ll see what happens.

And then within those bounds, users should have a huge amount of control over how the AI behaves for them because different users find very different things offensive or acceptable or whatever. And no company, certainly not either of ours, I think, is in a place where they want to, or should want to say, here are the rules.

Right now we just — again, it’s very new technology. We don’t know how to handle it. And so we’re being conservative. But the right answer, here, is very broad bonds, set by society, that are difficult to break, and then user choice.

And also, having a vibrant community of people who are trying to press on the limits of the technology, to find where it breaks and where it does unusual, interesting things, is good.

kevin scott

It’s super good.

sam altman

It’s really — it’s the way the security community works. I would rather have criticism, and people finding bugs in my software, than having those latent bugs in software get silently exploited, and causing harm. So it’s painful sometimes, to hear these things that are coming at us. But it’s not something that I, actually, am wishing that it goes away.

kevin scott

Sam, there are people, including some at OpenAI, who are worried about the pace of all of this deployment of AI, into tools that are used by billions of people, people who worry that maybe it’s going too fast, that corners may be getting cut, that some of the safety work is not as robust as maybe it should be. So what do you say to those people who worry that this is all going too fast for society to adjust, or for the necessary guardrails to be put in?

sam altman

I also share a concern about the speed of this and the pace. We make a lot of decisions to hold things back, slow them down. You can believe whatever you want, or not believe, about rumors. But maybe we’ve had some powerful models ready for a long time, that for these reasons, we have not yet released.

But I feel two things very strongly. Number one, everyone has got to have the opportunity to understand these tools, the pluses and minuses, the upsides and downsides, how they’re going to be used, decide what this future looks like, co-create it together. And the idea that this technology should be kept in a narrow slice of the tech industry, because those are the people who we can trust and the other people just aren’t ready for it, you hear different versions of this in the corners of the AI community.

But I deeply reject that. That is not a world that I think we should be excited about. And given how strongly I believe this is going to change many, maybe the great majority of aspects of society, people need to be included early.

And they need to see it, imperfections and all, as we get better, and participate in the conversation about where it should go, what we should change, what we should improve, what we shouldn’t do. And keeping it like hidden in a lab bench, and only showing it to the people that we think are ready for it or whatever, that feels wrong.

The second is, in all the history of technology I have seen, you cannot predict all of the wonderful things that will happen, and the misuse, without contact with reality. And so by deploying these systems, and by learning, and by getting the human feedback to improve, we have made models that are much, much better.

And what I hope is that everything we deploy gets to a higher and higher level of alignment. We are not — at Microsoft OpenAI, we are not the companies that are rushing these things out. We’ve been working on this and studying this for years and years. And we have, I think, a very responsible approach. But we do believe society has got to be brought into the tent early.

kevin scott

Are you worried about the kind of arms-race quality of this, that — Google is having an event this week. They’re reportedly rolling out a conversational I called Bard, and doing some updates to Search. Every other company I talked to, in Silicon Valley, is racing ahead to catch up with Chat GPT. Does that quality worry you?

sam altman

We aren’t the ones rushing out announcements, here. Again, the benefits of this technology are so powerful that I think a lot of people do want to build these things. But there’s totally going to need to be industry norms and regulation about what standards we need to meet on these systems. I think it’s really important.

casey newton

I’m curious if you could describe, even just a vague way, what sorts of capabilities you’re exploring, that feel a little bit too powerful to share, or haven’t been properly tested? Because this stuff feels —

sam altman

This is really talking out of both sides of your mouth.

casey newton

Well, how do you mean?

sam altman

Well, you’re saying, are you nervous about accelerating things, and can you tell us something crazy.

casey newton

Well, of the reason I ask is because I imagine some people are going to listen to this. And they’ve used Chat GPT. And they think this is really powerful. And they probably think it’s really cool. And they know that you all are also working on some more stuff. But it does sound, I don’t know, mystical or spooky, that there’s these other technologies we’ve been working on, that are even more powerful.

sam altman

No, I don’t — yeah, another downside of not putting this stuff out right away is, I think people assume that we have a full AGI, ready to push the button on, which is nowhere close.

We have somewhat more powerful versions of everything you’ve seen, and some new things that are broadly, I think, in line with what you would expect. And when we are ready, when we think we have completed our alignment work and all of our safety thinking, and worked with external auditors, other AGI labs, then we’ll release those things.

But there are definitely some major missing ideas between here and super-powerful systems. We don’t have those figured out yet. Nor, as far as I know, does anybody else.

casey newton

We’ve been hitting you pretty hard on the safety and the responsibility questions. But I wonder if you want to sketch out a little bit more of a Utopian vision here, for once you get this stuff into the hands of hundreds of millions of people, and this does become part of their everyday life, what is the brighter future that you’re hoping to see this stuff create?

sam altman

I think Kevin and I both very deeply believe that if you give people better tools, if you make them more creative, if you help them think better, faster, be able to do more, build technology that extends human will, people will change the world in unbelievably positive ways. And there will be a big handful of advanced AI efforts in the world.

We will contribute one of those. Other people will contribute one. Microsoft will deploy it in all sorts of ways. And that tool, I think, will be as big of a deal as any of the great, technological revolutions that have come before it, in terms of means for enabling human potential.

And the economic empowerment, the creative and fulfillment empowerment that will happen, I think it’s going to be — it could be jaw-droppingly positive. We could hit a wall in the technology — don’t want to promise we’ve got everything figured out. We certainly don’t. But the trajectory looks really good.

kevin scott

And the trajectory is towards more accessibility. The thing that I come to, over and over again, is the first machine-learning code, that I wrote 20 years ago, took a graduate degree, and a bunch of grad textbooks, and a bunch of research papers, and six months worth of work. And that same effect that I produced back then, a motivated high school student could do in a few hours, on a weekend.

And so the tools are putting more power in the hands of people. So now, it’s not even that you have to have a Computer Science degree to do very complicated things with technology. You can approach it —

sam altman

You can program in natural language, which is super cool.

casey newton

Yeah, Andrej Karpathy said, I think, that the next big programming language is English.

kevin scott

Yes.

Yeah.

casey newton

Which is very cool.

sam altman

Just think about it. That’s a thing, I think, that is holding us back so much, in terms of unlocking people’s creativity and potential. I grew up in rural, Central Virginia. People are smart and entrepreneurial and ambitious. You give them powerful tools, they’re going to go do interesting things.

kevin scott

Yeah.

casey newton

That’s very cool. And I know we’re out of time. But thank you guys for doing this. Really appreciate it. Thank you guys.

sam altman

Thank you so much. We appreciate it.

[THEME MUSIC]

casey newton

Coming up after the break, Google puts the AI in fail.

kevin roose

That’s really good.

So we have this big, Microsoft event. We get on a plane we come back to San Francisco. And then on Wednesday, Google joins the party.

casey newton

Yeah. And this was, I would say, the second time this week that Google tried to upstage Microsoft with AI-related announcements.

kevin roose

Right. So on Monday, Google announced its AI chat bot, its Chat GPT competitor, called Bard. And Bard is based on a version of LaMDA, which is Google’s language model. And Bard did not go so well in its first demo. In fact, in its first demo —

casey newton

Wait, wait, wait, wait, wait. We actually can’t call it a demo because unlike Microsoft, unlike OpenAI, Google still has barely let anyone actually try this thing. All that Bard is, at this point, is a blog post. And so in order to have a successful pseudo launch of Bard, Google really only had one job. And that was to not screw up the one screenshot of Bard, answering a question. So that was the task ahead of them, Kevin. How did they do?

kevin roose

They did not do so well. So Google shared a sample question that was asked of Bard, in an animated GIF. And — do you say GIF or JIF?

casey newton

I say GIF.

kevin roose

Yeah, GIF. This is a GIF podcast.

casey newton

Yeah, it’s a GIF podcast. The G in GIF stands for Graphical.

kevin roose

Settled. So this GIF showed Bard, answering the question, what new discoveries, from the James Webb Space Telescope, can I tell my nine-year-old about? Bard offered three answers, including one that stated that the telescope took the very first pictures of a planet outside our solar system.

So pretty quickly, a bunch of astronomers on Twitter pointed out that this is wrong, that the first image of an exoplanet, which is what a planet outside of our solar system is called, was taken in 2004, not by the James Webb Space Telescope.

Now there’s subsequently been some debate and saying, well, maybe Bard was actually right because there is this one, specific exoplanet that the James Webb Space Telescope was the first telescope to take a picture of. So if you read it one way, it actually was right. And we made the mistake, as humans, in interpreting it.

But I think the way that most people interpreted that, is that Bard was wrong. So in its first demo — not even demo — in its first screenshot —

casey newton

In it’s first screenshot — [LAUGHS]

kevin roose

Google’s AI chat bot made an error.

casey newton

So lots to say about this. But when I read this headline, I immediately flashed back to a conversation I had with a former Googler recently. And we were talking about this Code Red inside the organization, to try to really galvanize energy around AI stuff. And I was asking this person how they thought it was going to go. And this person said, the thing about Google is it doesn’t do well when it is panicked.

And this person reminded me of the last time Google panicked, which was when Facebook was really encroaching on their territory. And they announced Google Plus. We all know how Google Plus went. And so look, I think it’s probably going to go better for Google, with AI. But oh, man, this is not the way to start.

kevin roose

Yeah, an inauspicious debut for sure. And then on Wednesday, Google held an event, which we did not attend in person, in part because it was in Paris.

casey newton

Yeah. We would have loved to have attend.

kevin roose

We could be strolling down the banks of the Sien right now, enjoying like a nice cup of coffee and some macaron.

casey newton

(IN FRENCH ACCENT) A croissant, a pain au chocolat and AI. Name a better morning. I dare you.

kevin roose

But we were not invited, and couldn’t have made it anyway because we were en route back from Redmond. But this event, which was live streamed from Paris, featured Google executives showing off some of Bard’s capabilities, and talking about AI improvements that they’re making to a number of other Google products, including Maps and Google Lens.

And while the Microsoft event, I think, generally raised investors’ hopes for Microsoft, this event, on Wednesday, by Google, seemed to have the opposite effect. In fact —

casey newton

No, it literally had the opposite effect.

kevin roose

Yes. So after the event, shares in Alphabet, Google’s parent company, fell about 7 percent,

casey newton

(LAUGHING) which is like, do you know how bad your tech demo has to go, for your stock to drop 7 percent? OK, this is probably a hiccup. I think it’s going to be fine. But it is really remarkable, in this moment, how not great this stuff is going for them.

kevin roose

Yeah, and it’s just such a stark contrast to last summer, if you can remember, when we were all talking about Google’s mysterious new AI technology, that an engineer left the company, got fired, because he claimed it was sentient. So we’ve gone from sentient, chat-bot mysteries to shares falling 7 percent because —

casey newton

It can’t answer a telescope question.

kevin roose

— it can’t answer telescope question, in less than a year. And I just think that’s a remarkable turn of events.

casey newton

It is. Also, I would say, looking through these announcements — and I should say, I have not had a chance to try any of these features. But none of this stuff is really making me super excited. The stuff we’ve been talking about today is stuff that will legitimately change work for millions, potentially billions of people. And the stuff Google announced this week is just not playing in the same league.

kevin roose

Yeah, it feels like incremental, small ball, some of it. And I will say, I think it’s a mistake to underestimate Google, here. They’ve got some of the best AI researchers and engineers in the world. They’ve got a huge, dominant lead in search, and other categories. And they have all the data and processing power they need.

So I think they’re going to get up this hill rather quickly. But it is just — I think one effect that Chat GPT has had on the tech industry, that I think is actually good, is that it has forced companies to put up or shut up. We are not going to be impressed by blog posts or research papers. Unless I can use a thing, it is not real to me.

casey newton

You know what I read today, that I had forgotten? In 2016, Sundar Pichai stands up on stage, at Google I/O, and he says, as of today, Google is an AI-first company. That was almost seven years ago, that he said that.

And so look, there are trade offs to launching more quickly. We have a lot of concerns about what letting this stuff loose on the masses is going to mean for everyone. And yet, we also do think it’s cool. And we’re using it to do stuff. And that stuff is making our lives better and easier.

And it is just shocking that the company that invented the T and GPT is still this far behind. And actually, my question to you is, if you were Sundar Pichai, and you have what you have, do you do a blog post and an event this week, knowing what Microsoft is about to show off?

kevin roose

Interesting. I think I would probably wait. I think I would probably wait until my thing is ready for prime time, for the masses, because again, if it’s not — if I can’t log on and try it myself, it’s not real to me.

casey newton

This is one thing that Apple has always really understood, is there’s no sense in previewing something. Even if you feel like all your competitors are six months or a year ahead of you, stick to your knitting. Finish the project. Say nothing about it until the moment when you’re ready to put it on sale, make it available for consumption. And if you do that, and the product is good, then you win.

kevin roose

Totally. What would you do if you were Sundar Pichai?

casey newton

Here’s what I would do. I would say nothing until Bard was ready to be shown to the public. It’s one thing if they just want to, I don’t know, give a reporter a quote saying, yes, we do have something. You’ll hear about it soon. I think that’s fine.

But I don’t want to see another blog post, I don’t want there to be another event, until the stuff is ready for it to be shown, in exactly the same way that we saw this week, because at the end of the day, what users really care about is which of these services is better, which can I use now, if it’s going to cost me something, what is it going to cost me? Those are the relevant questions.

The relevant question is not, is Google working on something that will be available someday? We know that. So say nothing, and it’ll go better for you in the end. And I think it’s also just quite important that companies are doing all of the necessary safety work on these powerful language models, before throwing them open to the public.

I am really worried, as I know a lot of people in the AI industry are, about this arms-race quality that I asked Sam Altman about, where it is the case that Google is sprinting to get the stuff out the door, and whether it comes this week or next week or six months from now, my priority, as a human being, is for them to do the necessary work of making sure that these things are not going to be misused and weaponized, because I think that is a real danger, especially as these systems get better and better.

kevin roose

Yeah. All right, now let’s go look and see what that Space Telescope has been up to, since we’ve been talking.

If nothing else, I learned what an exoplanet is today.

[POP MUSIC PLAYING]

casey newton

Coming up after the break, what is the deal with bias in AI?

All right, Kevin. So have you been watching this Twitch stream called, “Nothing Forever?”

kevin roose

I have not.

casey newton

So it’s an art project from a couple of guys who had this idea, what if you could take generative AI, like the kind that powers everything we’ve been talking about today, and instead of using it to answer questions about telescopes and planets, what if you could make an episode of “Seinfeld” that just went on forever?

kevin roose

So it’s an never-ending, procedurally-generated episode of “Seinfeld,” where all of the lines are being written by AI, read by an AI voice, and there’s AI animation happening in the background?

casey newton

That’s right.

kevin roose

So both the dialogue and the video are being created in real time by AI? There’s no human, animating this stuff?

casey newton

Yeah. It’s not technically “Seinfeld” because that would be a copyright violation. And so instead of calling the character Jerry Seinfeld, the lead comedian in the show is called Larry Feinberg.

Everyone else has different names. Cosmo Kramer Zoltan Cackler.

kevin roose

No, come on.

casey newton

Yeah. Anyway, it’s a very low-quality, pixelated scenes, bad, middy music, and the action just shifts from place to place, from Jerry slash Larry’s apartment, to all the other locations in the “Seinfeld” universe.

And the characters just talk to each other. There’s a laugh track. There are some attempts at jokes. It will cut to the comedy club where Larry is doing stand up. And people are just really struck by the weirdness of this. It is occasionally funny. It went viral because the characters appear to become self aware for a minute —

kevin roose

Oh, god.

casey newton

— asking what they were all doing here. Actually, let me just play that for you.

[BAD DRUM MUSIC PLAYING]

archived recording

Did you ever stop and think that this might all be one big, cosmic joke?

Well, I don’t think it’s necessarily all — you know.

Yeah, I know. I just mean, why are we here?

To tell jokes, obviously.

I mean, why are we here together?

Maybe fate put us in the same place for a reason?

[LAUGH TRACK]

To make the world a funnier place?

Yeah, that could be it.

All right, I’m in. Let’s do it.

kevin roose

(LAUGHING) OK. So the vibe is sort of “Simpsons” meets Minecraft. But it’s got some elements of Seinfeld. It cuts between exterior shots of an apartment building and a Minecraft-looking replica of the apartment from “Seinfeld.”

So clearly, it’s trained on some “Seinfeld” episodes. But it’s also — the dialogue is weird. The voices are all robotic. The graphics are very bad. So it’s not like — I wouldn’t watch this for more than a few minutes. But I do think it’s really interesting as art experiment.

casey newton

Yeah, and we should say, there is a genius to this. So much of people’s TV consumption is just watching “The Office” from the start of season one until the very end, and then restarting it. If you could take the humor of that show, and generate endless riffs on it, in a never-ending episode of “The Office,” you could probably make a lot of money.

kevin roose

Someone just pitched that to Peacock.

casey newton

Yeah.

So anyway, unfortunately, as does sometimes happen with these AI stories, this one does not have a happy ending. Recently, the Twitch stream was suspended after one of these comedy stand-up segments, when Larry Feinberg, the virtual comedian, said some things that were homophobic and transphobic.

kevin roose

No.

casey newton

And they got —

kevin roose

Larry.

casey newton

— yanked. Yeah. And it does seem like this is a place we’ve been before. It seems, like often is the case, that you will let an AI chat bot have a little bit of room to run. And next thing you know, that they are reflecting all of the biases in our society.

kevin roose

Do we know why it became transphobic and homophobic?

casey newton

We do. And it’s super interesting. The dialogue in “Nothing Forever” was running off of GPT 3. And there are different flavors of GPT 3. The more sophisticated one, which “Nothing Forever” was using most of the time, is called Davinci. But it was causing some problems. And so the creators switched to a less-sophisticated model of GPT 3, known as Curie.

And the creators think that they accidentally stopped using Open AI’s content-moderation and safety tools when they switched over to Curie. So it was after they switched models, that the characters started saying the bad things.

kevin roose

That’s really interesting. How long had this thing been running before it broke bad?

casey newton

For a couple of months.

kevin roose

A couple of months?

casey newton

Yeah, it started up in December. And I think — I’ll speak only to the gay part of it, as a gay man. So one of the things that it said was, Larry Feinberg is musing about some comedy routines he might do in the future. And he proposes one that, quote, “all liberals are secretly gay and want to impose their will on everyone.”

And we should say, don’t be doing conspiracy theories about the gays. That’s bad. But that’s not the worst thing I’ve ever heard said about gay people. And I’m not saying that to let the AI off the hook. We do want these things to have controls. And we don’t want them to be running wild.

But I guess what I’m saying is, what I’m struck by is not, oh, my gosh, this AI just said the absolute most offensive thing that you’ve ever heard. I think what’s interesting is, it’s showing us how important it is to know what model you are using.

And here’s what I actually think that this story goes from a silly thing that happens online to something that has implications for anyone, working on this stuff. One of the stories I’ve been most fascinated by — we’ve talked about it on the show — is how CNET started to use this AI to generate articles, to provide SEO bait.

And one of my frustrations, in trying to understand the story, is CNET will not tell us which AI tools it is using. And so it becomes very difficult to evaluate how much we should trust this AI in the future because we literally don’t know what we’re using.

The way all of these tools work is, they’re trained on some body of material. And then they just predict answers based on that body material. Because that’s the case, it’s really important to know what the body of material is.

If you want to create an episode of “Seinfeld” that goes on in perpetuity, it would be really good to just train it on, I don’t know, every episode of “Seinfeld.”

kevin roose

Right.

casey newton

Right? And maybe some extra supplemental material. And if you did that, you would probably have a pretty good chance of not saying something super transphobic or homophobic.

kevin roose

Right. And it’s also going to be a test for platforms that host user-generated content. and just to link this with the Microsoft visit, like the Microsoft executive who used Bing to write a LinkedIn post, there’s not that much different from doing that, than creating an endless “Seinfeld” episode.

casey newton

Yeah.

kevin roose

We are entering a world where a lot of the content that gets submitted to social-media sites and posted online, that gets streamed on Twitch and YouTube, a lot of it is going to be algorithmically generated. And maybe there will be a human overseeing it and editing it, on the back end, before it goes up. But maybe there won’t. Maybe we’re headed into a world of never-ending “Seinfeld” episodes.

casey newton

Well, you said to me yesterday, that you assume that, going forward, any email you receive will not have been written by a person.

kevin roose

Oh, yeah. I’m assuming that we have reached the singularity on this already. It’s going to become increasingly hard to tell whether the post you see on Instagram, or the email in your inbox, or even the video that you’re watching on Twitch or YouTube, was generated by a human or not.

And as Microsoft was demoing this new writing assistant, that it’s built into its web browser now, I was thinking, I may never get a PR email from a human again.

casey newton

All right. So what do we do about all of this? I think that one of the answers, here, is just making people disclose what models they are using. And I’m going to say it. If you’re sending me email that was written by an AI, I actually want to know that. I don’t think I’m going to get that in Gmail. But I’d like it.

If I’m watching a stream, on Twitch, that’s generated by an AI, I do think it should disclose, this show is running off of this model. I want to be able to click on that model. I want to know where the training data came from. I want to know if it has moderation guidelines. We already have these community standards in place for all of these tech platforms. That’s been really good for their businesses. I think these large-language models should have similar kinds of disclosures because as you pointed out, this stuff really could damage trust for a lot of reasons. You never know who you’re talking to. We could rebuild that if we knew what technologies were baked into all these new tools we’re using.

kevin roose

Well, but to take the other side of that, when you watch a movie that has CGI graphics in it, they don’t tell you what program they used to create the CGI. We don’t expect that the things that we see on screen are all generated by human animators and directors. We accept that there’s some computer wizardry in there too. So how is that any different from getting an email from a PR firm, that was written by a robot?

casey newton

Well, when Marvel uses a computer program to make Iron Man fly, that’s just a piece of art that is designed to stimulate and entertain me. When you are using it to write emails, the assumption that everyone is trading on is that a person wrote this. And if that’s not going to be true, then I just think that we should have some basic questions answered for us.

kevin roose

Yeah. Yeah, I can see there being — when your email program puts a little thing at the bottom, that says, sent from my iPhone, please excuse typos or whatever. I think we will start to see some programs that put little disclaimers in there, like this email was drafted by Bing, or this email was drafted by GPT.

casey newton

Yeah, because what else is going to happen, is just in the same way that fake “Seinfeld” broke bad, these AIs, that people are using to write emails, are going to say stupid or offensive things, so that people are just scrambling, like, oh, it wasn’t me. It was the AI. And if you actually have a disclosure in there, you’ll have more plausible deniability.

kevin roose

Exactly. I can say all kinds of rude and offensive things over email now. And if people question me about it, I can just say it was Bing’s fault. Thanks, Bing. [MUSIC PLAYING]

casey newton

All right, well, I’d love to end this on a “Seinfeld” joke. But I really haven’t watched it much over the past 20 years.

kevin roose

Before we go, I just want to thank our listeners. We’ve been getting so many great emails, including a number that urged us to check out this endless “Seinfeld” episode that we just discussed. So thank you to everyone who wrote in, suggesting that we take a look at that. Please keep sending us your ideas.

casey newton

We really are listening to you. I also wanted to shout out our reader, Robin, who wrote, “in one of your latest episodes, you mentioned CNET’s false AI article on compound interest. I was struggling in my high school econ class, and stumbled across the article before it was corrected. As you can probably guess, I didn’t do so hot on my latest tests.

[laughs]

I feel kind of silly for not spotting the inaccuracies.”

Robin, don’t be so hard on yourself. You were misled by a rogue AI. But hopefully, you’ve now learned to be on alert.

kevin roose

Yeah, and please don’t use Bard for your next astronomy quiz.

[THEME MUSIC]

casey newton

“Hard Fork” is produced by Davis Land. We were edited this week by Jen Poyant and Paula Szuchman. This episode was fact checked by Caitlin Love.

Today’s show was engineered by Alyssa Moxley. Original music by Dan Powell, Elisheba Ittoop, Marion Lozano, and Rowan Niemesto. Special thanks to Hanna Ingber, Neil Gallagher, Kate LoPresti, and Jeffrey Miranda. That’s all for this week. The solution for Wednesday’s Wordle was flail.

Add a Comment

Your email address will not be published. Required fields are marked *