Inside the Hunt for the Discord Leaker, and Twitter Chaos Updates - The World News

Inside the Hunt for the Discord Leaker, and Twitter Chaos Updates

This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors. Please review the episode audio before quoting from this transcript and email [email protected] with any questions.

casey newton

I’ll tell you something that went on for me this week. For a long time, I’ve used the Google Docs product. Have you used this product?

kevin roose

The Google Docs?

casey newton

Yeah, you see this one? You see this one? OK, so —

kevin roose

Little, obscure, buried, you know —

casey newton

We like to highlight —

kevin roose

— feature.

casey newton

— up and coming apps. So I’m in the Google Docs product yesterday. And I’m working with some folks for a project I’m excited about. And these people are Google Docs masters, OK? I didn’t even know that Google Docs could be mastered because it looks so basic, but they keep saying things like, we’ll just go down to where I am, or go up to where I am. And I was like, bro, I don’t know where you are on the doc, OK? Like, I don’t have some sort of Doc ESP.

kevin roose

You gotta click the circle!

casey newton

You can click the circle! You knew you could click the circle?

kevin roose

Yes!

casey newton

OK, I have to tell people this. If you are in a Google Doc with other people and you’re editing collaboratively, you can just click the little circle at the top of the page with their little initial or their picture, and it will take you to wherever they are in the document.

kevin roose

Wow, I can’t believe you’re just learning about this in 2023.

casey newton

How long would you say you’ve known this?

kevin roose

At least five years.

casey newton

There’s no way.

kevin roose

Yes.

casey newton

Five years?

kevin roose

Yes.

casey newton

Why has this never come up before? This is insane to me. You know why it is? Because it’s in “The New York Times,” there’s like five bylines in every story and six editors. And so you actually have to know. I work for normal media organizations where you’re lucky if you could get an editor to come into the newsroom to help you out. And so maybe there just haven’t been that many people in my Google Docs.

kevin roose

Wow. Casey, I’m going to blow your mind. Pro tip, you can print a document on Google Docs.

casey newton

Now we’re just saying things that no millennial would ever do, but I think it’s great that Google has added these features. Generally speaking, you don’t want other people in your Google Doc, and I’ll tell you why — because they’re going to have ideas, and those ideas are going to make more work for you. But if you find yourself in this nightmare situation of working with other people, you can click the circle.

kevin roose

Wow, I love the idea of a podcast where every week, we just teach you how to use one extremely basic Office product.

casey newton

You know, you’re laughing, but I think that that could actually be quite useful to our readers.

kevin roose

Next week on “Hard Fork,” Casey learns Excel.

casey newton

We’re going to need more than one episode for that.

[MUSIC PLAYING]

kevin roose

All right, I’m Kevin Roose, tech columnist at “The New York Times.”

casey newton

I’m Casey Newton from Platformer.

kevin roose

And you’re listening to “Hard Fork.”

casey newton

This week, how a private Discord server ended up at the center of a national security crisis. Then, more chaos and clarity at Twitter. And finally, what happened when researchers gave AI memories? I’m guessing it was nothing good. We’ll find out.

[MUSIC PLAYING]

All right, Kevin, can we talk about national security for a minute?

kevin roose

Absolutely.

casey newton

All right, is this a secure line?

kevin roose

[LAUGHS]: Yes, I am in the sensitive room inside my house where I handle classified information. So we are good to talk about this stuff.

casey newton

Let me tell you, every room in my house is a sensitive room, depending on the day. But what I want to talk about is the national security aspects of it all.

kevin roose

Yeah, so this is a big military intelligence story that is currently making front page headlines in “The New York Times” and other publications. So on Thursday, federal investigators arrested a man named Jack Teixeira, who’s a 21-year-old Air National Guardsman, and they believe that he is linked to this leak of classified military intelligence documents. These documents have been circulating on the internet for a while now. And it wasn’t clear who had leaked them or why, but these documents really did cause a stir.

They have compromised some of our relationships with other countries. They have exposed weaknesses in the Ukrainian military. This is a big deal. And it seems to have stemmed from a very strange, very online source. So before we talk about the online piece of this, let’s just talk about what these documents were and why it’s important that they’re out there.

casey newton

Yeah, so these documents were, one, just very politically sensitive. Two, this was also supposed to be top secret, and there is stuff in there that the United States was trying to hide, both from its adversaries and allies alike. It included things like details about secret American and NATO plans for building up the Ukrainian military, for example.

And then third, sometimes when we hear about these leaks, they’re years old, right? Somebody finds them in a file cabinet and blows a whistle. The information in these leaked documents is, in some cases, weeks old. And it seems like stuff has been leaking for even longer than that. So that’s just a really big deal.

kevin roose

Yeah, and I would say this leak of classified information has really shaken the national security apparatus. So military intelligence officials have been looking into it furiously, trying to figure out where it came from and who did it. The State Department and other agencies are trying to handle some of the geopolitical fallout. And Merrick Garland, the United States Attorney General, made a statement on Thursday about the case.

casey newton

That’s right, and what’s really interesting to us is how these documents have been circulated. These documents were found on Discord servers, and not just any Discord servers, but video game Discord servers. And at least, at first, it seemed like people were posting these documents not to blow the whistle, but to win arguments online. And so, Kevin, when you and I heard that, we said, well, we need to get to the bottom of this.

kevin roose

Totally, and before we get into the details of these documents and why they’re being shared inside these peculiar Discord servers, let’s just explain for people who maybe don’t know what Discord is.

casey newton

Yeah, so Discord is a communications app. It’s free. And while it started as something that gamers primarily used, now basically everyone’s use it. And there are Discords devoted to posting memes. There’s Discords devoted to making generative AI. I have a Discord for my newsletter where people who read my newsletter can just kind of hop in and talk about today’s issue and other things that are on their mind. So it’s just kind of a very fast growing, very popular next generation social network, I would say, that is used by all sorts of folks.

kevin roose

Right, so I would compare it to something like Slack or Microsoft Teams. But it’s not work focused. It started, as you said, as a gaming thing. And there are different kinds of Discord servers, as they’re called, right? So anyone can start a Discord server. And some Discord servers just have a handful of people in them, but I’ve been in ones where there are also millions of people. So it can range from very small group chats to very large message board type things.

casey newton

Yeah, and an interesting thing about Discord is that you typically belong to both public and private Discords at once. So you might have a very small essentially a group chat with your friends, and then you also belong to a bunch of larger servers. And this ends up playing a key role in the plot here because what it seems like happened was that some materials that were designed to be shared only with a group chat pretty quickly migrated to some much bigger servers.

kevin roose

Yeah, so we are still piecing together exactly what happened with this leak of classified information. But what we do know is that the leak appears to have started on a private Discord server that was populated by a group of friends who played video games together. The server at one point was called Thug Shaker Central, which is — do you get that reference? I don’t.

casey newton

Yes, I looked it up, and I would say it is a meme that is not great.

kevin roose

Like, how not great?

casey newton

Like, I would say that this meme originated in the world of pornography, and I would say pornography with a somewhat racist bent.

kevin roose

Oh, OK. So we won’t talk about the origination of the name Thug Shaker Central, but that was the name of this Discord server. A bunch of friends are in there, talking about video games, presumably talking about lots of other things. And at some point, someone in this Discord server, who we now believe is the person who was arrested, the Airman, Jack Teixeira, shares these classified documents in this essentially group chat. And this might have just remained in this group chat, but because this is the internet, because this is Discord, because, as you mentioned, lots of people belong to multiple Discord servers, things just got really weird and out of control.

casey newton

Yeah, and look, I’m sure you’re in many group chats, Kevin. I know our listeners are, too. And if there is one lesson you can take away from this story, it is that stuff is actually going to get outside of the group chat.

kevin roose

Yeah, what happens in the group chat doesn’t stay in the group chat.

casey newton

That’s right.

kevin roose

And we don’t know exactly what happened in this case, but it appears that some of this classified material was eventually taken from this Discord server, Thug Shaker Central, and shared to a public Discord server that was devoted to a niche YouTuber called Wow Mow. Have you heard of Wow Mow?

casey newton

I have now heard of Wow Mow. I’ve heard of Wow Mow now. But before now, had not heard of Wow Mow.

kevin roose

[LAUGHS]: How now, Wow Mow. And this is a thing. Like, lots of YouTubers have their own associated Discord servers, where their fans will go and talk about their new videos or just talk about whatever. It’s sort of this fan community. So these documents, these classified military intelligence documents, end up in this niche YouTube Discord server for Wow Mow. And then from there, it seems like somebody in that server saw those documents, saved them, and then re-uploaded them to another Discord server, this one a big one devoted to Minecraft.

casey newton

And this really is the funniest part of the story because what appears happened is that in the midst of a minor spat, one of the people who had access to these documents just threw them into the chat by saying, here, have some leaked documents, which I think we can now agree is just a funny thing that we will now start saying to each other all the time.

kevin roose

It is funny, but it’s also like, this is a serious leak, right? This is leading every major news organization right now. The State Department and other agencies are scrambling to contain the geopolitical fallout from this. So this is a major event. And the fact that it has such a weird and sort of hyper-online origin story just made it totally fascinating to talk about.

And so I thought we should bring in one of the reporters who has been really leading the charge on this story. And that person is Aric Toler. Aric is a reporter for Bellingcat, which is a group that specializes in what they call open source research, or just basically using publicly available internet data to kind of solve mysteries and do reporting.

He teamed up with reporters on the visual investigations team at “The New York Times.” And he is an incredibly good internet sleuth. And he was actually one of the reporters who broke this story in “The New York Times.” So I thought we should get him in and talk to him about where this leak came from and what he can tell us about how this strange online Discord culture has birthed a true national security scandal.

[MUSIC PLAYING]

Aric Toler, welcome to “Hard Fork.”

aric toler

Hi there.

kevin roose

So you are part of a team of reporters that have been digging into this Discord leak and just published a story in which the name of the alleged leaker, or the leader of the Discord server where these groups were originally leaked is this 21-year-old from the Massachusetts Air National Guard named Jack Teixeira. So my first question is, how did you and your colleagues track this person down?

aric toler

Yeah, it’s a little convoluted, and there’s a lot of different platforms that most people probably know about, maybe not everyone. So this whole thing came to a head when everyone realized what was going on when “The Times” publish a story about how the Pentagon is looking into this leak. And at the time, they had mentioned that there were some posts on the messaging app, Telegram.

And so I was looking back, like, there’s no way this is the original source. It had to come from somewhere. And I looked and looked, and I found three photos and two other ones that weren’t in the Telegram cache on 4chan. But four of the ones on Telegram weren’t on 4chan, so clearly, they got it from the same place. I just don’t know where that place is.

So I looked and looked and looked — couldn’t find anything.

And then somebody DM’d me of like, hey I think I saw this on Discord. And they pointed me to a Minecraft map server. It’s where people make maps, of Earth. They try to make real life Earth on Minecraft.

And this guy had posted 10 on there. And of the seven that were posted on Fortran/Telegram, those seven were from this cache of 10. And there were three additional ones in here that no one had seen before, as far as I know.

So I talked to that Minecraft guy, very nice Minecraft guy. And he’s like, no it’s not me, because he was terrified for good reason. He’s like, no, he’s also a kid. These ever by the way, this is going to be a recurring theme, they’re all kids, a bunch of 17-year-olds. And this kid was like, no, I’m not the leaker. Everyone no, please everyone leave me alone. I’m innocent. I got this from a different server, a different guy. And he got them from another 17-year-old, another kid who posted them on another server about this YouTuber guy.

kevin roose

Yeah, so you’re of course talking about the WoW mouse server.

aric toler

Yes, so from there I talked to a few other people, four people in total, from the server. And eventually I figured out from these people I was chatting to that it was a different server that this kid was a part of that he pulled from and then posted them on the Wow Mao.

So there were hundreds and hundreds on this other server called — It had a lot of different names, but the name at the time of the leak was Thug Shaker Central, which is like this meme from a couple of years ago. And this is where hundreds and hundreds of these were posted on here.

And the 17-year-old kid took 107 of these and cross-posted them onto the Wow Mao server. And they’ve been getting posted on this older server, this Thug Shaker server since at the latest, October, it could have been earlier, but at the latest around October of last year.

kevin roose

So you followed the trail of breadcrumbs back to this server, Thug Shaker Central —

aric toler

Yeah. I didn’t think you’d say that out loud before.

kevin roose

And then how did you figure out who in that server had probably leaked these documents.

aric toler

Yeah, and from there I was like, OK well, there are only about 20 or so active members on this server. So the pool is pretty small. And I know because I talked to the kid who also was 17, and he told me about, yeah, so they call him OG. That’s not what anyone else called him by the way. He — just because I was the original leaker. And he’s like, well good, let’s just call him OG, the original guy, or the original gangster, or whatever.

kevin roose

So this literal 17-year-old who’s in this Discord server called Thug Shaker Central tells you that the person who originally posted this in that server, the leaker we’re assuming, is someone that he was calling OG.

aric toler

He was like, yeah, there’s this guy, you call me OG. He has money and he’s well-armed. He’s this really cool dude. And we played lots of games together on Steam.

So Steam is a gaming platform. It’s equivalent to the PlayStation store or the Xbox Game store. But he’s like, we played Arma 3 together, and we played Project Zomboid, and we played Halo. And he mentioned all these games they were playing. And that’s how they’re buddies and they just game together.

So it’s a bunch of teenagers, not just America, they’re from all over, who are on this little Discord server, about 20 active users or so. And then comes this guy who we now know is Jack. He came in and was sharing all these documents. Like, hey, look look what’s going on guys, that sort of thing.

So I knew a handful of people who were involved. So I knew that these guys all played Steam together. I knew some of the games they played, and all that. So I used a few different tools to find, because obviously these people almost all made their accounts private or deleted their account, basically everywhere after the story broke.

So there’s a couple of sites where steamid.uk, for example, that has historic scrapes of steam data, not just who their current friends are, but who their deleted friends are. Either people deleted their account, went private, or removed them from their friends list.

And I looked at some of these people’s accounts. And I’m like OK, here’s 400 friends, here’s 350 friends, here’s 300 friends. And who are the intersections? Who’s Who’s common friends with all of them? And basically all the people who were common friends with all the that I knew about, were people who are also in this server.

So like well, which of these is the most likely guy? And there’s one guy in particular who stood out. I looked a little bit more into this guy and I found — Actually, one of my colleagues over at Bellingcat was looking into his ID and they found that he was on this gun kit selling website. He used one of his Steam usernames he also used for selling some kit, like scopes and body armor, and that sort of stuff.

So we look more into this guy’s Steam list, and we saw, I guess, a few things that matched up. All the games that this guy, who’s on the server, told me they played together Arma 3, Project Zomboid, Counter-Strike, he played all of those games. There were screenshots he had posted, there was activity he played those games.

And so then I went through the usernames that he had for Steam, because you can change your username over and over. And the very first username he had was something, it’s like very different than the other, it was Jack something something tex. That sticks out, because the other names are like the Excalibur effect, very dramatic names.

And so I googled this name. And there’s two results. One, Steam profile, and two was this Flickr album, showing a guy and his kid. And this is from 2007. And it was Jack and Jacky, was the name of the album, for example. So I was like, well Jack is the name on the Steam ID. And but 2007 was pretty old.

So at first we thought that this dad was the leaker. But then we look more and more and we realize that we looked into the mom. And from there we found some other family members.

And we found photos of a new Air Force Reserve person, whose name was Jack, who had just recently entered into the Air Force National Guard. And he had shit posting adjacent stuff on some of his Instagram and his Facebook. And I was like, Oh this makes way more sense.

And then we looked at it more and we saw that one of the usernames for his Instagram and his Facebook was also something linked back to his Steam account too. So we realized that he just hijacked his dad’s steam account. So the Steam username and the Flickr username are the same.

That’s because, imagine you’re 12 or 13, and you play your dad’s Steam account. But you don’t want to make a new one because you already bought games on the account. You don’t want to make a new account where you lose access to all your Counter-Strike and PUGB and all that stuff.

So he just basically took over his dad’s account, and renamed it, and just started using it. But these sites retain the historic name info. So the dad’s Flickr account and the steam account were the same, which led to the family, which led to photos of him, and it linked back to the username he used for his Instagram and his Facebook were the same as for his revised, hijacked Steam account. And then from there realized, Oh, he’s actually with the Intelligence wing of this Air National Guard, and then everything fell together.

kevin roose

Wow. And besides deleting the server, the Thug Shaker Central server off of Discord, did the leaker try to cover his tracks in any other ways?

aric toler

He made his Steam profile private, but there are services out there, steamid.uk, big shout out to those guys who run that server. I’m a $9 a month patron subscriber so I got access to their premium stuff to help with this story.

kevin roose

So Aric, this is a very impressive bit of internet sleuthing, and I know that you have spent a lot of time tracking down not just this leaker, but many other people and incidents on the internet, that’s your specialty. But I think the details of this are surprising to a lot of people, just because it seems like such a mismatch. Usually when you have a leak of military intelligence, it’s coming from someone with an axe to grind, or a point to prove, or a whistleblower who has some conscience-driven reason that they want to get this information out there.

And it just seems so strange that it emerged in this Discord chat Thug Shaker Central, where these gamers, many of whom were literally teenagers, were just hanging out, trading insults, and memes and whatever else. So help us understand, for people who have not spent as much time in these Discord servers as you have. What is the culture of a server like Thug Shaker Central that would make someone, like this alleged leaker Jack Teixeira, want to post classified military documents.

aric toler

So again, they’re just 20 or so kids, and it’s a very intimate environment, because it’s voice chat too, you’re doing a lot of voice chat, you’re playing games, and all that stuff. So it’s like this very, I don’t know, sanitized, neutral, a lot of times like Twitter, or whatever. It’s a little bit more distant, where you see tweets and you interact with them very publicly. This is a lot more intimate. You have voice chat, things like that. So because of that people probably felt like they were very close to one another.

The kid who gave the interview to Washington Post said, he was my best friend. He felt he was very tight, very close to him. I’m not sure if it was reciprocated, but you can get the idea that there was this sense of they were a tight-knit community. So think of it as a more intimate group chat, I guess you could say, rather than a Discord server, which makes it sound like it’s a lot more official than it really was, which builds the trust I guess.

kevin roose

I guess what I’m wondering is, I’ve been in some gaming group chats. I have friends that I through playing video games. I have never once thought to leak them secret information from my job.

aric toler

You’re too old to think that, yeah.

kevin roose

But zooming out, as a person who has been studying extremism and online mischief for a long time, who has specialized in tracking down hard to find people and things on the internet, was it surprising at all for you that this new leak that is really turning the United States government upside down, was it surprising to you that it came from such a seemingly strange source?

aric toler

Not really. If you think about how is this league going to happen? Because I’m sure this happens. I’m sure that there are people who have classified documents who share them with their buds, like hot gossip. But most people don’t have a 17-year-old friend who will then post them onto a different Discord. So maybe the people who are doing this right now just have better friends, or at least maybe friends with better filters or sense of judgment.

kevin roose

Well yeah, one lesson I’m taking from this is that people are just not as good at doing crimes as they think they are. And that, especially when the internet is involved, they just do not understand how to share things with other people in a secure way.

aric toler

Yeah, it’s insane.

kevin roose

Word do the gamers out there, careful.

casey newton

I feel like we’re all learning how to share with each other in a secure way in this life, Kevin.

kevin roose

Aric, thank you for coming on Hard Fork.

aric toler

Sure. Thanks for having me. [MUSIC PLAYING]

kevin roose

Casey, it has been several weeks since we have talked about Twitter on this show.

casey newton

We had to set it aside for a minute and let it cool down.

kevin roose

Yeah.

casey newton

Yeah.

kevin roose

It went into timeout.

casey newton

Yeah.

kevin roose

But now it’s back —

casey newton

Yeah.

kevin roose

— and a lot has been happening. So I have to confess that a few months ago, I was doubting whether all the predictions of Twitter’s doom are actually going to come true. People kept saying the app is dying, people are leaving, it’s going to break, they’re laying off all these people. It still ran and seemed pretty normal. But now it looks like things are seriously starting to decline.

casey newton

Yeah.

kevin roose

And we don’t have time to run through all of the disasters. So I just picked three categories of Twitter chaos that I want to ask you about, checkmarks, circles, and clown shit.

casey newton

The three C’s.

kevin roose

So let’s start with checkmarks.

casey newton

OK.

kevin roose

So we’re talking about the checkmarks which are officially called verified badges. And Elon Musk, he said that on April 1st, every person who does not pay for Twitter Blue, who has one of these checkmarks, members of the media, or celebrities, or politicians would lose their checkmarks. That date came and went.

casey newton

Yeah.

kevin roose

And as of right now I’m pulling up my Twitter profile and I still have a checkmark.

casey newton

Oh look, I do too.

kevin roose

So what is the deal? What is going on with Elon Musk and these checkmarks?

casey newton

Well, look on one level checkmarks did play an important function on the old Twitter, because they would tell you that somebody was who they said they were. And that helped with impersonation. And then Elon decided that the best path forward for Twitter as a business was to pivot it away from advertising and towards subscriptions.

And the main carrot that he wanted to offer to people in order to get them to subscribe was a verification badge of their own. As you well know, the first round of this went very poorly. They let anyone buy a badge for $8. They didn’t check to see who anyone was. People impersonated the pharmaceutical company Eli Lilly, and many other institutions at some point, causing wild swings in their stock price.

And so it got put on a back burner. And Elon said, let us think this through a little bit more. And so yeah, as you say, the new deal was going to be, OK, everybody April 1st. If you’re not paying, you’re going to lose your badge. And then that got delayed. And so now he’s saying everyone’s going to lose their badges on April 20th —

kevin roose

That’s 4/20 —

casey newton

— the weed day.

kevin roose

Lots of stoners out there. [LAUGHING]

casey newton

And we’ll see. Elon has a very ChatGPT-like relationship with the truth. This is somebody who is not drawing on a store of facts all the time, I would say. And so when I hear that he has set a new deadline, no part of me believes that it’s real. I think it could happen. But I don’t have any expectation that it will.

kevin roose

Right. What do you think is actually going to happen on 4/20, the new day when all the checkmarks are supposed to disappear from these legacy-verified people and organizations.

casey newton

Well here’s what we know. So far only a very small percentage of Twitter users are paying for Twitter Blue, which includes this badge. He keeps extending the deadline, I think, because he assumes people are going to fall in line, and they’re going to say, Oh, I’ve gotten my extension, I guess I’ll get that 8 bucks together and I’ll finally pay.

And he’s just banking on the fact that these badges are going to become more precious to people. I don’t think that’s going to happen. So at some point he’s going have to put his money where his mouth is. And he’s just going to have to to start taking away people’s badges, and then see if that has the desired effect.

I don’t think it’s going to. But I can also see why he would want to delay it, because after he does that, he doesn’t have a lot of moves left. You could imagine moves, he could start charging people to tweet, or something like that. But if we get to that point it’s going to be in a very desperate place. So I can see him delaying this again and again.

kevin roose

Yeah. My bigger question is we know that Elon Musk wanted to shift Twitter from an ad-based model to a subscription model, that is the business problem that he is trying to solve by instituting this $8 a month Twitter Blue verification scheme. Does the fact that this program has been so unpopular mean that people just aren’t willing to pay for social media at all?

casey newton

I think people are willing to pay for all sorts of things on social media. It tends not to be these features that were previously offered for free. But TikTok was the highest grossing app last year. And the reason it isn’t because you have to pay for your badge.

So there’s a lot of money out there for the social media ecosystem, but you have to bring real product and design chops to it. You have to offer a stable, functioning product. And Elon just hasn’t been able to deliver on either of those.

kevin roose

Yeah, to me it shows the power of anchoring. If you start a new social app and you say, this is going to cost $8 a month, and here are all the things you get, or you can give tips to your favorite creators, that seems like a proposition that people might actually go for. But if you take something that used to be free and then start charging for it without real increase in value, people are anchored to that free price, and it’s going to seem not just a stretch for them, but it’s going to seem offensive to them.

They’re like, I used to get this for free, I’m giving you my tweets for free. You’re using them to make money. Like, I’m not going to pay you for the privilege of filling up your platform with my content.

casey newton

Totally. And to me, the exciting thing here is this is a move that is finally making people start to say, you know what, maybe I should stop giving this guy my tweets for free.

kevin roose

All right, next C on the list, circles. Twitter Circles are a product that I have never used. What are they?

casey newton

So Twitter Circles is very similar to the Close Friends feature on Instagram. So if you use Instagram, you can create a list of your very close friends, and then if you post to your story, these ephemeral posts that only lasts for 24 hours, those stories will only be seen by people in your Close Friends.

Twitter circles launched with the same basic idea, that instead of broadcasting your tweet to the entire world, you could broadcast to a subset of the people that you follow. Now I never use this feature very much myself, because most of the people that I would include in my Close Friends are media people. And if I said something stupid on Twitter to my Close Friend circle, they would just screenshot it and get me fired.

kevin roose

This is a huge relief to me, by the way, because I was paranoid that you did have a Close Friend circle on Twitter, and that I wasn’t in it. I was posting detailed reviews of your podcast performance every week to a close group of friends.

casey newton

No, I wasn’t using it. But I’ll tell you, I have some gay friends in particular who were using circles to post let’s say some spicier, more adult-oriented content. And so that made what happened on Twitter Circles this week something of a catastrophe.

kevin roose

OK, what happened.

casey newton

So what happened was, as I said, these posterior circles are supposed to be limited to a group of people that you’ve put on an allow list. But then some poor souls browsing the Twitter feed started to see posts on the For You page. The For You page is the algorithmically-ranked feed of posts that Twitter might think you would enjoy for any number of reasons. it’s ranked, but it could potentially be seen by anyone.

And so essentially you had private posts that were being, not just made public, but recommended to the public. So the best analog I can offer here is, imagine that you posted something really heartfelt, or embarrassing, or vulnerable on your Close Friends Instagram story. And then a stranger opens up the Explorer page on Instagram and saw it promoted at the top of their feed. That’s what just happened on Twitter Circles. And in a way it’s funny to me that this hasn’t gotten more discussion, because this is a truly catastrophic privacy breach.

kevin roose

Right. It’s like if your DMs were just showing up —

casey newton

Yes.

kevin roose

— on people’s Twitter timelines. It’s like that level of a privacy scandal.

casey newton

Yeah, absolutely. Elon did an interview this week, where he boasted that, well, a lot of folks at Twitter would be dead by now. But it’s not dead, because his standard for his Twitter alive, I think, is can you still log into the website? But this is a great example of how when you get rid of a huge number of people that work on your technical infrastructure, weird stuff is just going to start breaking in ways that can have quite a terrible effect on your user base.

kevin roose

Yeah, I’ve been wondering whether there will be some mass DM leak that stems from all of these engineers getting laid off. That hasn’t happened. But this Twitter Circles thing does seem to be pretty close. These are supposed to be private, but all of a sudden started showing up on people’s For You feeds.

casey newton

Oh yeah, and when the layoffs started, Twitter engineers started messaging me saying, don’t use DMs anymore. They’re just not a safe product. So I don’t use Twitter DMs anymore for really anything remotely sensitive. But yeah, watch out man.

kevin roose

Wow, so do you have any idea of what went on behind the scenes to make these Twitter Circles posts start showing up for people they weren’t supposed to be seen by?

casey newton

To my knowledge, they have said nothing about it, which does not exactly inspire confidence.

kevin roose

So put your spicy tweets somewhere else, because they are not safe in Twitter Circles.

casey newton

Yeah, OnlyFans is now accepting spicy, private subscription-only posts.

kevin roose

When are you starting your OnlyFans?

casey newton

Well, we’ll see how Substack goes.

kevin roose

OK, that brings me to my third catchall category for what I am calling, clown shit. This would include things that Elon Musk has done in the past couple of weeks that just seem like he’s just playing for laughs, or not like behaving in a serious way. He briefly changed the Twitter logo to the Doge image, the Shiba Inu dog, that is also the face of Dogecoin. That lasted for a few days. It was not on April Fool’s Day.

casey newton

No, I believe it launched on April 3rd. And everyone was joking Twitter tried to do an April Fool’s joke. But it took an extra two days to launch, which seems plausible to me.

kevin roose

Yeah, totally. He also painted the sign on the Twitter building in downtown San Francisco, or he had it painted, so that it just now reads Titter. I truly don’t believe that even an 8th grader would find this stuff funny. I can’t imagine anyone is observing any of this and thinking, this is so funny and cool.

This is the man that I want to entrust my company’s advertising budget to. He also changed his display name on Twitter for a day or two recently to Hairy balls, Harry spelled, H, a, r, r, y. And balls spelled, B, o, l, z.

casey newton

This man is not well. Can we just say, the man is not well.

kevin roose

Yeah, what is going on?

casey newton

Look, I think that driving a social network can actually drive you crazy. It’s very difficult. You’re running a platform that is hosting this huge portion of the world’s speech. You’re getting criticism from absolutely every corner. And that’s hard enough even if you are a very good CEO with a good plan for where you’re going.

I think if you come into that with no background in the subject matter, and you’ve committed $44 billion, and you are losing money every month, and none of your ideas are working. Oh, and also you’re working on a handful of other giant companies that have their own problems, that is just not a recipe for success. And so many people Elon took over said, well look, this guy is going to be different. He’s special, he builds rockets. A social network, yeah, I think he can handle that. And then you fast forward to April 2023, guess what honey, it didn’t work.

kevin roose

He’s like, give me the rockets back.

casey newton

That’s right.

kevin roose

Right, this isn’t rocket science, it’s actually harder.

casey newton

It is harder. But is there one person who wouldn’t be relieved if Elon Musk says, I am stepping away from Twitter, and I’m just going to work on rockets full time. Everyone would be overjoyed.

kevin roose

I think even his fans would be overjoyed, because he’s been getting blowback from some people who previously have supported him, people like Paul Graham who is a big Silicon Valley investor. He’s a co-founder of Y Combinator. And he’s been criticizing Elon recently for basically just taking his eye off the ball, wasting his time running Twitter instead of working on SpaceX or Tesla.

So even the people who are sort of predisposed to support him are not fans of what he’s doing at Twitter. I want to ask you about one more thing that I would put in the clown shit category, which is, what is going on between Twitter and Substack?

casey newton

Well so, ethics disclosure, I publish my newsletter platform on Substack. And they’ve helped me in various ways over the years, and I have a note on my website about that. But Substack decided, in the midst of all this chaos, to —

kevin roose

And Substack for the people who may not know what it is. It is a newsletter platform. You can subscribe to newsletters either free or paid. Substack is just the infrastructure that powers that and allows people to pay you to read what you have to write.

casey newton

Yeah, that’s right. And they saw everything going on at Twitter. And they said, well gee, it seems like this thing is having some trouble. Maybe we could build something similar of our own. And so they built this product called Notes. It launched this week. It lets you post text, and links, and pictures. And you can reshare those. They call that restacking.

And when Elon got word of this, he really overreacted. And for a time he blocked users on Twitter from engaging with tweets that had links to Substacks in them. If you click the link, it would say, this website is spammy or dangerous.

kevin roose

Which is true in the instance of Platformer —

[INTERPOSING VOICES]

casey newton

Platformer is a spammy and dangerous newsletter.

kevin roose

But for other Substack newsletters, it’s better. But yeah, it’s unfair.

casey newton

So they mark Substack links as, quote, “potentially spammy or unsafe. They also block searches for Substack. So if you search the word Substack, it would just show you results for the word, newsletter. So it’s like, Oh, you’re looking for a Substack. Well, we’re not to show you that, but we will show you newsletters. So when he got asked about this, Musk said that Substack was quote, “trying to download a massive portion of the Twitter database to bootstrap their Twitter clone. And I went over to Hacker News to see what they were saying about this comment, which I found completely inscrutable. And the Hacker News people, this is a community of engineers, and some of the most delightful nerds you’ll ever meet.

And the Hacker News people were just laughing their heads off. Because they were like, what does this even mean? It’s like you can’t download it. It just didn’t make any sense whatsoever. And so, as usual with him, there’s not a relationship with the truth there, it’s just like spewing words, and it was all nonsense.

kevin roose

What I found most interesting about this beef between Elon Musk and Substack, is that one of the things that animated him when he took over Twitter, was that he thought that the previous management of Twitter was throttling or shadow banning sources of news that they didn’t like. He was very upset about what happened with Hunter Biden and that story during the 2020 election, where Twitter blocked people from sharing that “New York Post” story about Hunter Biden.

casey newton

Right.

kevin roose

And then he Hunter Bidened Substack. He made it so that you could not share Substack links, or comment on them, or like them, or search for them. He disappeared Substack from the platform.

And Substack is where a lot of the people that he is ideologically allied with, these Twitter files journalists, and other opponents, and skeptics of the mainstream media are going. He was basically cutting off his friends and allies from one of their main sources of distribution.

casey newton

It truly is the funniest possible outcome to this story. And I just think the more serious point is, this is not a principled man. This is someone who talked about nothing except his speech principles and his desire for bipartisan dialogue, and let’s move to the center.

And he spoke about it in the most grandiose terms. And the minute he felt the slightest bit of discomfort, all those principles went out the window, and he said, I am going to punish my enemies. And look, some people are very upset about this. I can’t get upset about it, because every day people learn what I truly have been saying all along, which is the emperor has no clothes here. And you can just go on Twitter every day now and see it for yourself.

kevin roose

But one of the other things that you have said is that this chaos at Twitter is going to result in people moving.

casey newton

Yeah.

kevin roose

So far that hasn’t really happened. There has been no mass exodus of Twitter users to another platform. You’re very active on Mastodon now. I think it’s safe to say that has not really threatened Twitter in a massive way. Now people on Twitter are talking about Blue Sky, which is this decentralized alternative to Twitter, that was started by Jack Dorsey.

But none of these alternatives to Twitter have really gained momentum. So what is it going to take for something to pop up and replace Twitter? Or is it just kind of like the entire concept of Twitter in our media ecosystem dies off and is replaced by, I don’t know, group chats, and Instagram accounts, and whatever else.

casey newton

So there’s two answers to that. There’s the product answer and there’s the NPR answer. I want to talk about the product answer first. So you just named some products that compete with Twitter in various ways. And you’re right, none of them has supplanted Twitter in the popular imagination. But again, let’s remember, Elon has only owned this website since October. We are six months out from when he took over. We first started talking about this on Hard Fork.

The reason that these products are now being built is because there is an appetite for them. They are growing. And, frankly, there are just now so many of them that even if one of them hasn’t entered that period of hockey stick growth quite yet, I do think it is probably going to happen for someone.

So you’re going to have to give it some time. Someone is probably going to have to invent some fun, new little mechanic. There’s going to have to be some weird, silly new toy that just captures our imagination.

But I do think it’s going to happen. So first answer is, these products are still getting built. But they are getting built, and that was not true before Elon Musk took over. Nobody smelled blood in the water until the past few months.

At the same time, Kevin, some important people actually are starting to leave. And I want to deliver one of our signature Hard Fork tributes this week to a little media organization called National Public Radio,

kevin roose

NPR, yeah I’ve heard of them.

casey newton

NPR they were just minding their own business. They were just tweeting links to their stories. And then one day they wake up and they’ve been slapped with this label, state-affiliated media. And state-affiliated media was a label that the old Twitter created to identify essentially propaganda.

kevin roose

It was like a little label that went under the profile of essentially foreign propaganda outlets, just to let that the tweets that you were seeing from this account have state-affiliation.

casey newton

Yeah, it’s just trying to give you a little bit more information so you know where the account is coming from. They allowed them to post, but they wanted you to know that. Well all of a sudden, NPR which does receive some minor amount of government funding, but is completely editorially independent of the United States government, they now have this label as well.

So it effectively said, this is propaganda. That was the message that I believe the people who added this label to NPR’S account wanted you to receive. And messages are exchanged.

And eventually the state-affiliated media label gets downgraded to the government-funded media label, even though it gets less than of its funding from the Corporation for Public Broadcasting. So the entire point if this was not to serve users of Twitter in any way. It was just to slag NPR and to try to make it look less credible.

kevin roose

It was a dunk.

casey newton

It was a dunk. And on Wednesday, Steve Inskeep tweeted a picture of John Lansing, who is the CEO of NPR, at a Morning Edition meeting, Morning Edition one of the flagship shows of NPR, and said that NPR will deemphasize Twitter. He said that aside from the misleading label, Twitter isn’t used by most Americans, drives little traffic to NPR, and no longer has the public service relevance that it once had. And to that I just want to say —

kevin roose

Slay.

casey newton

Well said. This is what I have been saying, and pretty soon a lot more people are going to get it too.

kevin roose

You think media organizations are going to leave Twitter?

casey newton

This is what I have been waiting for, is for somebody to wake up and say, I do not want to come to a platform every single day. And get punched in the face. You know what I’ve been thinking about lately, Kevin? Do you remember during the Trump campaigns, when there would be these rallies. And in the center of the rally there would be a pit for the media. And a signature moment of every rally would be Trump pointing to the people in the media so that everyone at the rally could say, boo, we don’t like the press.

That is what Twitter has become. It is the press pit, where a bunch of people are standing around you in a circle, jeering. Adding this state-funded media badge was one of those steps. But I’m barely joking when I say that I think eventually every reporter who is still on the service will have a clown badge next to their username. And you just have to decide if you still want to be there when it happens.

kevin roose

So you are still on Twitter.

casey newton

Well in the way that I started with. so in December I was like, I got to change the way that I’m using this website. And so I basically started using it as an RSS feed.

Whenever I have a Twitter-like thought, I put it on Mastodon. And when I have some new piece of journalism to share, a newsletter a podcast, I put it on Mastodon, and then I repost it to Twitter. I’m not in there having fun, I’m really not replying to people.

I’m reading it, because it feels important to know what is being said on Twitter, and it’s important to know how Twitter is falling apart. Social networks are my beat, I have to pay close attention to it. But I am not really doing free labor for Elon Musk anymore.

kevin roose

Fascinating.

casey newton

But you still are. Let’s talk about that.

kevin roose

Well I want the clown badge. That would be fun for me. OK. so Twitter is in clown town right now. But next I want to talk about a different town. One that is populated entirely by AI.

casey newton

Ah, oh.

kevin roose

We’ll be right back

[MUSIC PLAYING]

Casey, did you watch Westworld, the HBO show?

casey newton

Yeah, I watched the first few seasons, but I gave up as it became increasingly incomprehensible.

kevin roose

OK, so I also watched one season, and I think it fell off because I just didn’t get it anymore, which I attributed to me being dumb, but I think maybe the show just also got worse.

casey newton

Well no, here’s how. It became one of those shows where at first it was very exciting to learn that person you thought was a human was actually a robot, but then that was just the entire show. It was like, well guess what other human is a robot? Every member of the cast.

kevin roose

Right, so for people who haven’t watched Westworld, the premise, is there’s this theme park where there are humanoid robots that you can go and interact with, and they’ll talk to you.

casey newton

But they perfectly lifelike.

kevin roose

Perfectly lifelike. But when the guests leave, the robots memories all reset. And so the next time you go, you can just have brand new interactions with them.

casey newton

Except what Westworld asks is, what if they didn’t actually forget?

kevin roose

Exactly. So I want to talk about this next story, because I think it is actually moving us in quite an interesting direction that is a lot like Westworld. So a paper came out last week by researchers from Stanford and Google. And the title of the paper is called Generative Agents Interactive Simulacra of Human Behavior.

casey newton

Another one of these catchy clickbait titles from the academics.

kevin roose

And I want to talk about this paper, because it’s really fascinating. And I think it actually points to the next phase of what’s going to happen in AI. Now we’ve been talking a lot on this show about these large language models, these chat bots, things like ChatGPT, and how you can actually use them as a persona.

You can say, you are my tutor, or, you are my speechwriter, or, you are my editor, or, you’re my friend. Talk to me as if you are that thing. And what these researchers wondered is, could you create a city, a virtual simulated environment, and fill it with AI-generated characters.

casey newton

A simulated city, or Sim City if you will.

[LAUGHING]

kevin roose

Right. Basically could you make the Sims, but the Sims are coming up with things on the fly. So the Sims is this very popular computer game. But in the Sims all of the interactions between characters are scripted —

casey newton

Right.

kevin roose

— and programmed. So what these researchers were wondering is, could you do the Sims with no scripting? If you just took a set of AI-generated personas, which they’re calling generative agents, and place them into a virtual environment, like a city, and just pressed play. What would they do?

casey newton

I have to say, very interesting question.

kevin roose

So the way this works, they created 25 characters. They put them in a virtual environment that they called, Smallville, which was an Animal Crossing-style video game. And they gave all 25 of these characters short personality descriptions.

So one example was a character named John Lin, and the thing that the researchers told the AI. And in this case, they did this using the ChatGPT API. So all this is happening with the help of ChatGPT.

So the thing that they put in as the description of this character was, John Lin is a pharmacy shopkeeper at the Willow Market and Pharmacy who loves to help people. He is always looking for ways to make the process of getting medication easier for his customers. John Lin is living with his wife Mae Lin, who is a college professor and son, Eddie Lin, who is a student studying music theory. John Lin loves his family very much.

casey newton

Sounds very sweet.

kevin roose

So they build these pre-programmed personalities for these characters. And then they set them loose. They just say, we’re going to observe you for two days. And we’re just going to see what these characters will do.

casey newton

And within the context of this game, are their objectives, are these characters moving throughout the world, are they just bumping into each other on the street and striking up conversations. What is happening?

kevin roose

Yeah, so the way it works, the researchers who are controlling this experiment can prompt the characters. They can say you are going to the store to get some food. Then the language model fills in what happens next.

casey newton

OK.

kevin roose

Or you’re a character, you’re bumping into another character in the park, and you’re going to have a conversation, what does that conversation say? And then the language model fills in the rest of the conversation, and any subsequent actions.

casey newton

Got it.

kevin roose

So I can actually show you what this looked like.

casey newton

I’ll see this.

kevin roose

I’ll share my screen here. So this is a replay of the Smallville. So you can see it’s like a blocky, pixelated map —

casey newton

Yeah.

kevin roose

— with some various rooms. And then you can scroll down here. And here is a list of 25 characters.

casey newton

They are these very cute little sprites.

kevin roose

Right. And so you can click on any the characters and see a log of all of their actions. So here’s a character named Ryan Park. And Ryan Park’s current action is sleeping. And he is in his apartment in the bedroom.

casey newton

And that’s goals. Cheers to you Ryan Park.

kevin roose

And here is another character named Sam Moore. Sam is taking a walk around Johnson Park. He’s stopping to take pictures. Oh, and he’s having a conversation with another character named Adam Smith. And all of this is being generated by the language model.

And along with these characters, the researchers actually developed something really interesting. That is like a virtual memory for these characters. Because one of the problems with language models, one of the limitations, is that they don’t remember things.

So you can have a conversation with ChadGPT. You can tell it some things, and then if you come back a different day and start a new thread, it’s not going to remember what you talked about. You have to reconstruct it to be a therapist, or a teacher, or a friend, or a negotiating counterparty, or whatever.

So they get around this with something that they call the memory stream, which is basically a natural language list, like a text list of what a given character has been doing and experiencing. Any conversation they’ve had gets stored in this memory file. And then periodically these memories get synthesized into inferences and reflections, summarized.

So you basically have a system where each of these 25 characters can refer back to a file that contains a synthesis and a summary of everything that has happened to them in this simulation. And they can refer to that in different contexts. So if one character is walking through the park and sees two other characters talking, they can infer that those people must be friends, and must know and each other. And then they can use that information later on in the simulation. So it’s a neat, technical trick that simulates a memory.

So these researchers they let this experiment run for two days. And they just observe what these characters do, prompted by this large language model. And what they found was really interesting.

So their first finding was that the I agents behaved in more believable ways than human agents. So they did a control group experiment, where after this two day period was up, the researchers asked the generative agents a series of questions to assess their self knowledge, their ability to make plans and remember things. They also gave that test to a group of 25 humans, who had access to all of the same information about these characters.

And they basically gave them a replay of what the characters had been doing for these two days and all of their memories. And so the humans answered those questions. And then the researchers had these other humans, these evaluators go in and rate how believable they thought those answers were. And the generative agents were evaluated as being more believable than the humans.

casey newton

So that’s bad news for humans. So go on.

kevin roose

So that was one finding. The other very strange finding that they had was that there were these emergent behaviors that they did not expect and did not intend to happen between these generative agents.

casey newton

OK, if you tell me that one of these things started asking if it was part of a simulation, I’m going to need a drink.

[LAUGHING]

kevin roose

No, they started leaking military documents. So one of the emergent behaviors was that these generative agents gossip.

casey newton

What?

kevin roose

They shared information with each other without prompting. You’re telling me that these Sims are spilling the Tea on each other?

These Sims are spilling the tea. So in one interaction, the researchers told one character, this character named Sam Moore, that he was going to run for mayor of Smallville. And they didn’t give him any more instructions, other than you are starting a campaign for mayor.

Without prompting, Sam more than shares this information with the rest of the town and starts a mayoral campaign. And the other agents start talking about this without like interacting with Sam Moore. They have learned this information.

casey newton

This is admittedly big news in a town where, for the most part, people are just sleeping and talking on a park bench. Like I could see why this would be the buzz in Smallville.

kevin roose

But not only are they talking about the fact that Sam Moore is running for mayor, but they’re sharing opinions of him. So one agent, at one point, says, I’m still weighing my options, but I’ve been discussing the election with Sam Moore. What are your thoughts on him? And another agent says, to be honest, I don’t like Sam Moore. I think he’s out of touch with the community and doesn’t have our best interests at heart.

casey newton

Well go off King. [LAUGHING]

kevin roose

And not only are they sort of gossiping about each other, but they’re also coordinating and planning together. So in one case, the researchers instructed an agent named Isabella to throw a Valentine’s Day party. That was the instruction that they gave this character. And this character, Isabella, starts spreading the word. And by the end of the simulation, 12 characters know about this party. And how many of them do you think showed up?

casey newton

In the real world, a lot of people would flake, but I’m hoping that these agents are more respectful. So I want to hope that all 12 showed up.

kevin roose

Well you are wrong, because these are believable entities. And much like humans, many of them flaked. Seven characters flaked on the party, three of them said they had other plans, and the other four just didn’t show.

casey newton

It’s like setting up a Grindr date. This is horrible.

kevin roose

But five characters actually did show up at this party. And this was not something that the researchers had predicted. So why are we talking about this you may be asking?

casey newton

Well I assume that each week you show up and try to terrify me with some new glimpse at our coming dystopia. Was there another reason?

kevin roose

So I do think this has dystopian implications. But I think the most obvious application for this is something like gaming.

casey newton

Absolutely, yes.

kevin roose

So in games a lot of them, especially roleplaying games, or open-world games, there are these things called NPCs, or non-playable characters. And they’re the kind of characters that you come across. And they tell you some line, or give you some hint. How would you describe an NPC.

casey newton

Yeah, an NPC is essentially an object in the game that exists to give you an objective, or to advance you along a quest. But they are scripted to make you feel like you are interacting with something sentient. And of course you know it’s not.

But the question that this poses is, well, what if word got around about what you were doing in this virtual world? What if that character could ask you about that? Or maybe change their behavior? This thing does already happen, but all of those interactions are scripted. What you’re suggesting here is that we might not have to script these things in the future, they can just happen.

kevin roose

Right, they could just act as the language model tells them to act. So one example that was sort of a joke, but I think someone said in response to this paper that you could be playing a game now, and you could kill an NPC in the game. And you could return back to that same spot later, and all of the other NPCs could be holding a funeral. And that’s not something that the game designers would have to necessarily script.

I did talk to the lead author of this paper, who’s a Stanford researcher named Joon Sung Park, and I said, well beyond games, how could this be useful? And he said that one of the ways that this could be used was to simulate behaviors in online groups. So for example, if you were going to start a new subreddit devoted to celebrity gossip, and you wanted to see how would a set of rules, given to users of this subreddit, encourage them to react? Would they become rebellious? Would the thing sort of devolve into a toxic cesspool? How could you basically simulate that without actually having to start the subreddit?

casey newton

Yeah.

kevin roose

And this experiment suggests that you could actually just start a simulated subreddit, and make some simulated users, and see what happens.

casey newton

Totally. Another thought that comes to mind is, I think we both believe that AI companions are going to be a really big deal. There’s going to be some kind of virtual agent that you’re talking to, whether it’s on your phone or your desktop.

And in order for that to work, they need to have a memory. They need to remember what kind of day you were having yesterday. They need to the names of your friends and family, and other people who are important to you.

What I think is interesting about this is, it suggests, what if you actually have multiple different companions for different purposes? You’re going to have your personal trainer and your therapist. And of course, at first, none of these are going to be as good as humans, but they probably all get better over time.

And what if they talk to each other and share information. What if you have a virtual doctor who remembers your entire health history, and does it make you write down your birthday every time you go to the office? So I think there is a huge amount of potential here, and that is a very fun glimpse into somebody already running the experiment.

kevin roose

So yeah. So this is the experiment that got a lot of attention this week. But there’s also this related thing that is happening in AI world right now, which is that a lot of people are trying to make these large language models into agents. So turning them from chat bots into things that can act autonomously without human intervention.

casey newton

We’ve talked a fair bit about the need for AI vocabulary, and I’m telling you, agent is just one of those words that you are going to start hearing absolutely all the time.

kevin roose

Yes, so in the last couple of weeks, there have been a number of projects that have tried to basically cut out the human. So right now. If you go to use ChatGPT, you prompt it, you type something in, it types something back. You type something in, it types something back. It’s like a ping pong.

And what these experiments with AI agents are asking is, what if you just cut out the human? What if the AI model could essentially talk to itself and come up with plans, and execute those plans, and then generate AI agents to carry out different parts of this task?

casey newton

Yeah, we are really just determined to wipe out humanity, are we?

kevin roose

Well some people literally are. So right now there’s a lot of research and energy in the AI world around turning these large language models into what are called agents. One of them is called Auto-GPT. And Auto-GPT is a tool that someone hacked together that allows you to, instead of just like typing to a chat bot, and it types back, you actually give it a task.

And the AI can then talk to itself and recursively execute that task, or try to execute that task. So you could tell theoretically this Auto-GPT to start a dropshipping business.

And it could then talk to itself and say like, OK for that, I’m going to need someone who’s an expert in shipping. I’m going to need someone who can talk to the big e-commerce companies. I’m going to need someone who can set up maybe a Stripe account. I’m going to need someone to code up a website. And also I’m going to need someone who can go on Twitter and advertise this business.

Now just to be clear, there’s no evidence yet that Auto-GPT can actually do this stuff well. It’s still not very good, and there’s still some pretty big technical hurdles standing in the way of these agents being able to actually do things autonomously. But that’s the direction that some people are trying to push these AI systems.

casey newton

Yeah, it’s like when you’re fighting a boss in a video game, and you think you’re finally whittling them down, and then they just summit a bunch of other demons to eat your face. That’s what we’re building.

kevin roose

Right. And the examples of this Auto-GPT. stuff that have been floating around are mostly just people trying to do these business tasks. But there is one project called Chaos-GPT —

casey newton

Oh no.

kevin roose

— where someone basically used Auto-GPT, and gave it the explicit instruction to try to destroy humanity.

[LAUGHING]

casey newton

Well the world is still intact as we record. So what happened?

kevin roose

So yeah, this has not worked as you can see by the fact that we still exist.

casey newton

My world was destroyed when Taylor Swift and Joe Alwyn broke up. But other than that.

kevin roose

But there is a mode on Auto-GPT called continuous mode, which basically allows it to, in theory, run forever, just doing these recursive task loops. And when you turn on continuous mode, it does actually pop up a little warning saying, you might not want to do this, because it is, quote, “potentially dangerous, and may cause your AI to run forever, or carry out actions you would not usually authorize. But the mode exists people are using it. It’s out there already.

And I’ve talked to some AI safety researchers in the last few days who are very concerned about this shift from thinking of these I language models as tools, to using them to create autonomous agents. And what they’ll say is, look, these experiments, these Smallville, these generative agents that are just going to parks and talking to each other, and planning Valentine’s Day parties, these are not dangerous, because they’re contained in this virtual environment, and they’re pretty safe in the things that they are talking about.

But you actually don’t want I agents operating without human oversight. You don’t want them carrying out their own goals. You don’t want them doing long-term planning and coordinating with each other. That’s just not a capability that you want to build.

casey newton

Yes, and while I’m glad that some of this research is being done in the academic realm, the fact that these experiments are already being run does make me hope that our government is paying attention, and is pretty soon going to start to have some ideas about whether and how all this ought to be regulated.

kevin roose

Yeah, I will say I’m pretty skeptical of regulation in this space, at least the proposals that I’ve seen so far. But when I saw Chaos-GPT which stated that its explicit goal was to destroy the world, I was like, yeah, that should probably be illegal.

And this is the thing, all the scenarios of AI doom out there imply that the robots are going to have to become power-seeking and greedy for some resource that it will kill everyone to get. But actually it may be that some troll on the internet is just going to give it an instruction to go destroy humanity.

casey newton

And as soon as that’s done, you know what the first thing they’ll do is? They’ll go on Discord and say, hey, look at this, guess I won.

kevin roose

Right. So the last concern that I heard from AI safety researchers about these AI generative agents, is that these things that start off as being very benign and not all that capable tend to get more powerful over time.

casey newton

Like the Hard Fork podcast!

kevin roose

Exactly. And so you can imagine hooking some of these generative agents up to physical robots. That is a thing that can be done.

casey newton

Like Megan?

kevin roose

Yes. You could take one of these generative AI personas and tell it to control a hardware robot. And it could do that.

casey newton

You know what? My dream of a sassy Roomba has never been closer to reality.

[LAUGHING]

So I really just think this is an area to watch, and that I hope will come back to on this show. Because this use of AI to create these autonomous agents, I think poses some really interesting opportunities. I do think there are applications for this beyond gaming. And I think it’s an area where there is the potential for a lot of harm and mischief. So stay tuned, we will be giving you more updates on the AI agents. But in the meantime, I think I want to go check out Smallville actually. I want to see what the buzz is. Are they talking about us? Are they listen to the show?

kevin roose

I don’t think so, but I’m going to check out another character, maybe we’ll find something here. Latoya Williams is currently taking a yoga class, doing standing poses.

casey newton

Everyone in this town is having a more relaxing day than I have in Dallas.

[LAUGHING]

kevin roose

Let’s go to Smallville.

casey newton

All right, let’s do it.

kevin roose

All right. [MUSIC PLAYING]

Hard Fork is produced by Davis Land and Rachel Cohn, or edited by Jen Poyant. This episode was fact-checked by Caitlin Love. Today’s show was engineered by Alyssa Moxley.

Original music by Dan Powell, Elisheba Ittoop, Marion Lozano, Rowan Niemisto and Sophia Landman. A special Thanks to Paula Schumann, Nell Gallogly, Kate LoPresti, Pui-Wing Tam, and Jeffrey Miranda. You can email us, as always, at [email protected] But please, no top secret documents. Save those for your Discord server.

[MUSIC PLAYING]

Add a Comment

Your email address will not be published. Required fields are marked *