Mon, 6 March 2023 at 11:25 am GMT
A couple of months ago, most people would probably have looked blank if asked if they had heard of artificial intelligence chatbots.
Of course, that was before the launch of OpenAI’s ChatGPT, which has become something of a media sensation. Depending on what you choose to believe, ChatGPT, which is touted as providing plausible-sounding text that may be indistinguishable from that written by a human, is either a valuable tool, something that provides a means for students to cheat when writing essays, or an entity that will rise up to destroy humanity, like Skynet in the Terminator films.
When faced with something that may go on to change the face of the world as we know it, there is one question that we think needs to be answered. Can it pass the ultimate test and solve the Telegraph’s Cryptic Crossword, which has challenged the minds of the brightest people for almost a century?
We decided to give it a go.
On the face of it, solving cryptic crossword clues should be meat and drink for ChatGPT. Many clues can be read as mathematical formulae, once you know some basic rules.
Most clues contain a definition of the answer, together with wordplay which indicates both the letters of that answer and how they are sequenced. This might mean taking letter-string X and sticking it on letter-string Y to come up with the answer.
In another clue, you might place X inside Y. Elsewhere, you might jumble X and Y. That sounds perfect for an AI to work out. Or so we thought.
The first clue we threw at ChatGPT was a straightforward one, where the clue contains two definitions of the answer:
Lay out large meal (6)
Pleasingly, the AI came up with the correct answer, SPREAD. However, its explanation left a lot to be desired, as it claimed that S is an abbreviation meaning “large” (it’s not; it means “small”), “meal” indicates BREAD, and taking S + BREAD gives SPREAD. Hmm. Still, we gave it the benefit of the doubt; all of us at one time or another have come up with the correct answer to a clue while not fully understanding how it works.
Solving the next clue, which also contained two definitions, wasn’t as successful:
German song told a tale (4)
Here, the solution should be LIED, which can be a noun meaning a German song, or a verb.
It was now that its inability to understand that the answer length is given in brackets at the end of the clue became clear, as it came back with BALLAD. Quite how ChatGPT worked out that B is an abbreviation meaning “German”, that “song” means LAD and that combining the two gives its answer, really isn’t clear.
Things didn’t improve when trying it with an anagram clue, even when we explicitly told the AI that the answer contains 13 letters, rather than relying on the number in brackets:
Performer, in revolt, quits unexpectedly (13)
The expected answer was VENTRILOQUIST, which is an anagram (unexpectedly) of IN REVOLT QUITS.
REBELLIOUSNESS, which has 14 letters. It’s worrying when the world’s most famous AI chatbot can’t count.
An unexpected success came with the following clue:
Remember Paddington? Popular personality (4,2,4)
The answer? BEAR IN MIND. Obviously even chatbots are fans of everyone’s favourite marmalade-guzzling bear.
It’s fair to say that, by the time ChatGPT had tried to solve all of that day’s clues, the results were disappointing; just seven out of 30 were solved successfully. Many of the bot’s other answers bore little resemblance to the intended solutions, which leads us to draw two conclusions.
Firstly, cryptic crosswords, at least for now, are still best solved by humans.
Secondly, and perhaps more importantly, if you’re ever faced with a death-dealing robot powered by a rogue AI, just ask it to solve an averagely difficult crossword clue. That should stop it in its tracks for a while.
Can you outsmart ChatGPT? Try your hand now at the famous Telegraph Cryptic Crossword and much more on our fantastic new puzzles website.