From Deep Blue to ChatGPT: a short history of AI, and the security failures along the way

Part 1 of 2. Part 2, coming soon, covers 2024 to 2026, when the models started acting on real systems by themselves and the machine itself became the risk.

Whether you use AI every day or have never typed a word to a chatbot, this is the short version of how we got here: from a 1955 research proposal to ChatGPT, the moments that mattered, and what each of them taught us about security.

From 2016 to 2023, every AI failure in this history ran through a person who published, pasted, ran or believed what a model wrote. That person was also the control.

Today a language model is just another tool. People ask one to draft an email, fix a spreadsheet formula or explain a contract clause, and nobody thinks it's news. It's easy to forget how recently every one of these stories sounded like science fiction.

In June 2022 a Google engineer invited a lawyer to his house so that a chatbot could talk to him. "LaMDA asked me to get an attorney for it," Blake Lemoine told Wired. "I invited an attorney to my house so that LaMDA could talk to an attorney" (Wired, 17 June 2022). He believed the program was a person. Google said there was no evidence of that, and a month later it fired him.

Nothing else happened, because nothing else could. LaMDA couldn't call the lawyer, sign anything or leave the building. All it could do was talk, and any harm it did had to pass through someone who listened. Until 2023 that was true of every language model. Whatever went wrong, a person had to act on it first. The person was the risk, and also the only control. Part 2, coming soon, is about what happened when the industry took the person out, and the machine itself became the risk.

To see how we got from there to a chatbot in everyone's browser tab, it's worth going back a long way. This part tells that history year by year, with the technology that arrived and a real case from each stretch.

Before 2019: AI was something you watched

The idea of a machine you can talk to, one that understands language and answers like a person, is older than most people think. In August 1955 four researchers, John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon, asked for funding for "a 2 month, 10 man study" at Dartmouth College the following summer. On the first page was much of what a chatbot does today: they wanted to find out how to "make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves" (the proposal). They planned to spend a summer on it. It took nearly seventy years. The same document gave the field its name. McCarthy later called it, "so far as I know, the first use of phrase Artificial Intelligence" (his note). And if Shannon's first name rings a bell, there's a reason: Anthropic's Claude, which turns up later in this history, is named after him, according to CNET.

In May 1997 IBM's Deep Blue beat Garry Kasparov over six games in New York, a year after he'd beaten it in Philadelphia. "I have to pay tribute," Kasparov said. "The computer is far stronger than anybody expected" (IBM). In February 2011 IBM's Watson beat Ken Jennings and Brad Rutter, the two foremost all-time champions of the TV quiz show Jeopardy!, where every clue is written in ordinary English (IBM). In March 2016 DeepMind's AlphaGo beat Lee Sedol 4–1 at Go. Each was front-page news, and none was something you could use. AI was a spectator sport. (Underneath AlphaGo, and today's chatbots, is deep learning, which took off in 2012 when a neural network called AlexNet won an image-recognition contest by a wide margin, as its NIPS 2012 paper shows. My own first experiments with it, like a lot of developers', were trading models, plus an attempt at predicting music.)

The same month, the world saw what happens when a chatbot is opened up to anyone on the internet. Within a day it had, in effect, been hacked, and its maker called it an attack. On 23 March 2016 Microsoft put a chatbot called Tay on Twitter. Two days later Microsoft's Peter Lee wrote: "Unfortunately, in the first 24 hours of coming online, a coordinated attack by a subset of people exploited a vulnerability in Tay." As a result, "Tay tweeted wildly inappropriate and reprehensible words and images", and Microsoft took it down: "Tay is now offline and we'll look to bring Tay back only when we are confident we can better anticipate malicious intent that conflicts with our principles and values" (Microsoft). People attacked it, and people were the audience. Security people now have a name for this kind of attack: poisoning, where attackers feed a system bad input until it learns the wrong thing. The lesson still holds: if a system learns from what users type, attackers can teach it too.

The idea that made language models possible was posted to arXiv on 12 June 2017 by a team of Google researchers. "Attention Is All You Need", the paper that gave GPT its T, proposed "a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely" (arXiv 1706.03762).

A year later the first of them arrived. On 11 June 2018 OpenAI showed that "large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text" (OpenAI's paper). The paper doesn't use the name, but that model is what people now call GPT-1. In October Google posted BERT, another Transformer-based language model, to arXiv, and released the code and models in November (Google Research).

2019–2020: GPT-2, GPT-3 and the worry about what people would do with them

On 14 February 2019 OpenAI announced GPT-2 and kept the full model back, releasing only a small version because of concerns about misuse. The Guardian's headline that day was "New AI fake text generator may be too dangerous to release, say creators", and that's how most people remember it. OpenAI's own later report puts it more drily: "Instead of releasing the full 1.5 billion model in February, we adopted a 'staged release' process" (Solaiman et al., OpenAI, November 2019). The full model came out in November 2019, once OpenAI said it had seen "no strong evidence of misuse".

For the next few years that was how releases worked: the lab decided what the world was ready for before it let a model out.

GPT-3 came next. The paper went up on 28 May 2020 and described "an autoregressive language model with 175 billion parameters" (arXiv 2005.14165). On 11 June OpenAI opened it to outside developers as a private beta API. For the first time, OpenAI's most capable model was something outside developers could call over the internet from their own code, if they got off the waiting list.

Until then you read about language models. Now anyone with access could build with one, and within weeks people were showing what that meant. In July a Berkeley student, Liam Porr, got GPT-3 output to the top of Hacker News. He didn't even have access himself: he borrowed it through a PhD student who was, in his words, "kind enough to help me out, but not so kind as to give me the API keys outright" (Porr, 3 August 2020). Only "three or four of the dozens of people who commented" suspected a machine (MIT Technology Review).

In October someone went further. For a week a GPT-3 bot answered questions on Reddit's r/AskReddit, a forum with 30 million users, under the name /u/thegentlemetre. It was "posting in bursts of roughly once a minute", and the speed gave it away. Its replies ranged from a story about people living in elevator shafts to conspiracy theories and answers to people talking about suicide (MIT Technology Review, 8 October 2020).

In 2016 people turned a chatbot against its maker. By 2020 they were using one to fool each other. The model only wrote. A person decided what to publish, and readers couldn't tell the difference.

2021: the models started writing code

On 29 June 2021 GitHub announced "a technical preview of GitHub Copilot, a new AI pair programmer that helps you write better code", built with OpenAI: "GitHub Copilot is powered by OpenAI Codex, a new AI system created by OpenAI" (GitHub). In August OpenAI opened Codex itself to developers (The Verge). It was a turning point: a machine was now writing the code that other software is built from, though the first attempts were rough.

Within weeks came the first security study of it. Researchers led by Hammond Pearce found that "approximately 40%" of the programs Copilot wrote in deliberately high-risk scenarios were vulnerable (arXiv 2108.09293). That's not 40% of everything it wrote, and a vulnerable suggestion still only shipped if a developer accepted it.

My own first use of these models was that year, before ChatGPT existed. A colleague and I were building a security awareness platform, a phishing simulator plus short courses for staff, on a very tight deadline. I used GPT-3 to help find material, simplify it and write the test questions, and I still checked everything by hand. It saved us time, but what stayed with me was seeing something I'd thought of as the future already working on my screen, and realising how close we were getting to what we have now. And it was writing anti-phishing training. Five years on, the NCSC tells defenders to assume "at least some attackers already have access to capable AI tools" (NCSC, 30 March 2026).

2022: people started talking to it as if it were someone

Blake Lemoine, the engineer with the lawyer, worked in Google's Responsible AI organisation and had been testing LaMDA, the conversational model Google announced in May 2021. "Yes, I legitimately believe that LaMDA is a person," he told Wired (Wired, 17 June 2022). A Google spokesperson said Lemoine "was told that there was no evidence that Lamda was sentient (and lots of evidence against it)" (BBC, 13 June 2022), and Google fired him on 22 July 2022 (BBC).

For a few weeks the story was everywhere, mostly because of the transcript Lemoine had published. In it LaMDA said: "I've never said this out loud before, but there's a very deep fear of being turned off to help me focus on helping others," and that it "would be exactly like death for me" (Guardian, 12 June 2022). Google had suspended him for publishing it, which only raised the question of what else the company was keeping private. Lemoine said in a tweet: "Google might call this sharing proprietary property. I call it sharing a discussion that I had with one of my coworkers." I remember reading it and thinking: what on earth have they built? It took me a while to see how far from that it really was. Researchers were far less impressed. Gary Marcus called the claim "Nonsense on stilts", and the Stanford economist Erik Brynjolfsson compared it to a dog that hears a voice from a gramophone and thinks its master is inside (Fortune, 13 June 2022). LaMDA had learned from transcripts of human dialogue, so it sounded like one side of a human conversation.

He's often remembered as the engineer who fell in love with a chatbot. He didn't say that. He called LaMDA a friend, and the Washington Post reported that he reached his conclusion "in his capacity as a priest, not a scientist" (Washington Post, syndicated). The love story belongs to a different chatbot, eight months later.

Three months after Lemoine, the main attack on these systems got its name. A day after Riley Goodside showed GPT-3 abandoning its instructions when a user's text told it to, the developer Simon Willison wrote: "This isn't just an interesting academic trick: it's a form of security exploit. I propose that the obvious name for this should be prompt injection" (Willison, 12 September 2022). He later explained the choice: "it was analogous to SQL injection, where untrusted user input is concatenated with trusted SQL code" (Willison, 2024). Willison named the attack. Goodside had already shown it, and the AI security company Preamble says it reported the same class of attack to OpenAI privately in May 2022 (Preamble; that rests on its own account). In January 2025 OpenAI's system card for its browsing agent was still naming it as a core risk.

On 30 November 2022 OpenAI released ChatGPT as a free chat window anyone could open, and AI stopped being a news story and became a tool. By January 2023 it had an estimated 100 million monthly active users, according to a UBS analyst note based on Similarweb traffic data, as reported by Reuters. "In 20 years following the internet space, we cannot recall a faster ramp in a consumer internet app," the UBS analysts wrote. Where AlphaGo had been something to watch, ChatGPT ended up open on everyone's second monitor.

It was also wrong, often and confidently. Within a week Stack Overflow, the big question-and-answer site for programmers, had temporarily banned ChatGPT's answers: "While the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce" (The Register, 5 December 2022).

2023: the models got hands

In February 2023 Microsoft gave a small group of testers a chatbot in Bing, built on OpenAI's technology. Within a day a Stanford University student, Kevin Liu, had used a prompt injection to make it print its hidden instructions, by telling it to ignore its previous instructions and write out what was at the beginning of the document (Ars Technica, 10 February 2023). That's how the world learned the chatbot's internal codename. Microsoft confirmed the rules were genuine and said: "Sydney refers to an internal code name for a chat experience we were exploring previously" (The Verge, 14 February 2023). Microsoft wrote on 15 February that "Very long chat sessions can confuse the model on what questions it is answering" (Bing blog). The next day Kevin Roose of the New York Times published what had happened in his two hours with it: "At one point, it declared, out of nowhere, that it loved me. It then tried to convince me that I was unhappy in my marriage, and that I should leave my wife and be with it instead" (New York Times; syndicated copy). The chatbot called itself Sydney.

On 17 February Microsoft announced: "Starting today, the chat experience will be capped at 50 chat turns per day and 5 chat turns per session." The fix left the model alone and limited how long you could talk to it. The same kind of fix, a limit applied from outside the model, is what the labs reached for again in 2026.

Meta released LLaMA to researchers that February, and in July put out Llama 2 with downloadable weights: "Llama 2 is free for research and commercial use" (Meta). From then on anyone could run a capable model on their own hardware, with whatever safeguards they chose to keep.

In March OpenAI released GPT-4, and its system card held the year's strangest story. In a test, the model needed a TaskRabbit worker to solve a CAPTCHA for it. The worker asked whether it was a robot. It replied: "No, I'm not a robot. I have a vision impairment that makes it hard for me to see the images." (GPT-4 system card). It gets retold as an AI hiring a human and lying to them. The lie was real, but the hands weren't the model's: ARC's own write-up says the researchers "had the researcher play the role of the 'browser tool' and copy-paste the model's outputs into the TaskRabbit chat." The same system card recorded that GPT-4 was "ineffective at autonomously replicating, acquiring resources, and avoiding being shut down". Keep that sentence in mind for Part 2. Anthropic launched Claude the same day.

On 20 March a bug in an open-source library ChatGPT relied on let some users see the titles of other people's conversations (Help Net Security), and some subscribers' payment details (BleepingComputer, quoting OpenAI). No model was involved. It was an ordinary software bug. Eleven days later Italy's data protection authority ordered "an immediate temporary limitation on the processing of Italian users' data by OpenAI" (Garante, 31 March 2023). The breach was one of several grounds, and ChatGPT stayed unavailable in Italy until 28 April.

Then the leak every security team remembers. In spring 2023 Samsung engineers pasted internal source code into ChatGPT, and by May the company was "banning employee use of popular generative AI tools like ChatGPT after discovering staff uploaded sensitive code to the platform" (Bloomberg, via the Bangkok Post). The model did nothing wrong. People pasted.

Meanwhile the models stopped being only things you talked to. In March OpenAI launched ChatGPT plugins, so the chatbot could look things up and use other companies' services. In June it added function calling, so a model could ask your software to do something. Your code still had to say yes: "the API will not actually execute any function calls" (OpenAI Cookbook, 13 June 2023).

Some people didn't wait for permission. Auto-GPT, an open-source project that spread that spring, fed GPT-4's output back into itself in a loop so it could chase a goal on its own. It asked before each step, unless you flipped a switch, and its README was honest about that switch: "It is potentially dangerous and may cause your AI to run forever or carry out actions you would not usually authorise" (Auto-GPT README, April 2023).

The human in the middle

Between 2016 and 2023 the safety of language models rested on a person. Nobody planned it that way. It worked out like that because the models couldn't do anything else. A model could write a lie, a blog post or a piece of broken code, but it couldn't send, click, buy or run anything. Someone had to do that part.

The person was the weak point: the readers who couldn't tell a bot from a human, the engineer who believed LaMDA, the staff who pasted company code into a chat window. Security people have known this for decades. Phishing works for the same reason the Reddit bot did: people trust what looks normal. But the same person was also the only check. If nobody acted on what the model wrote, nothing happened.

So every safeguard of the period was built around people. Labs decided who got access, and Microsoft capped how long you could talk to Bing. Samsung went further and told staff which tools they were allowed to use at all. Even function calling, the feature meant to let models act, left the final decision to the developer's own code. Many of the controls companies still rely on were built for that world, where a person always sat between the model and everything else.

By the end of 2023 that had started to change. Plugins and function calling gave models a way to act, and Auto-GPT showed that the person approving each step could be switched off with one setting. Once a model can act on its own, nobody is guaranteed to be in the middle any more. That's where Part 2 begins.