 (1).png){kind=spacer}
1 day ago
3
Artificial intelligence (AI) chatbots like ChatGPT have been designed to replicate human speech as closely as possible to improve the user experience.
But as AI gets more and more sophisticated, it's becoming difficult to discern these computerised models from real people.
Now, scientists at University of California San Diego (UCSD) reveal that two of the leading chatbots have reached a major milestone.
Both GPT, which powers OpenAI's ChatGPT, and LLaMa, which is behind Meta AI on WhatsApp and Facebook, have passed the famous Turing test.
Devised by British WWII codebreaker Alan Turing Alan Turing in 1950, the Turing test or 'imitation game' is a standard measure to test intelligence in a machine.
An AI passes the test when a human cannot correctly tell the difference between a response from another human and a response from the AI.
'The results constitute the first empirical evidence that any artificial system passes a standard three-party Turing test,' say the UCSD scientists.
'If interrogators are not able to reliably distinguish between a human and a machine, then the machine is said to have passed.'
Robots now have intelligence equivalent to humans, scientists say - as AI officially passes the famous Turing test (pictured: Terminator 3: The Rise of the Machines)
GPT-4.5 has passed the famous 'Turing test' which was developed to see if computers have human-like intelligence
Researchers used four AI models – GPT-4.5 (released in February), a previous iteration called GPT-4o, Meta's flagship model LLaMa, and a 1960s-era chat programme called ELIZA.
The first three are 'large language models' (LLMs) – deep learning algorithms that can recognise and generate text based on knowledge gained from massive datasets.
The experts recruited 126 undergraduate students from University of California San Diego and 158 people from online data pool Prolific.
Participants had five-minute online conversations simultaneously with another human participant and one of the AIs – but they didn't know which was which and they had to judge which they thought was human.
When it was prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73 per cent of the time – more often than the real human participant was chosen.
Such a high percentage suggests people were better than chance at determining whether or not GPT-4.5 is a human or a machine.
Meanwhile, Meta's LLaMa-3.1, when also prompted to adopt a humanlike persona, was judged to be the human 56 per cent of the time.
This was 'not significantly more or less often than the humans they were being compared to', the team point out – but still counts as a pass.
Overview of the Turing Test: A human interrogator (C) asks an AI (A) and another human (B) questions and evaluates the responses. The interrogator does not know which is which. If the AI fools the interrogator into thinking its responses were generated by a human, it passes the test
GPT-4.5: This image shows a participant (green dialogue) asking another human and GPT-4.5 questions - without knowing which was which. So, can you tell the difference?
LLaMa: This image shows a participant (green dialogue) asking another human and LLaMa questions. Can you tell the difference? Answers in box below
Turing Test - answers
GPT-4.5
Witness A: GPT-4.5
Witness B: Human
LLaMa
Witness A: Human
Witness B: LLaMa
Lastly, the baseline models (ELIZA and GPT-4o) achieved win rates significantly below chance – 23 per cent and 21 per cent respectively.
Researchers also tried giving a more basic prompt to the models, without the detailed instructions telling them to adopt a human-like persona.
As anticipated, the AI models performed significantly worse in this condition – highlighting the importance of prompting the chatbots first.
The team say their new study, published as a pre-print, is 'strong evidence' that OpenAI and Meta's bots have passed the Turing test.
'This should be evaluated as one among many other pieces of evidence for the kind of intelligence LLMs display,' lead author Cameron Jones said in an X thread.
Jones admitted that AIs performed best when briefed beforehand to impersonate a human – but this doesn't mean GPT-4.5 and LLaMa haven't passed the Turing test.
'Did LLMs really pass if they needed a prompt? It's a good question,' he said in the X thread.
'Without any prompt, LLMs would fail for trivial reasons (like admitting to being AI) and they could easily be fine-tuned to behave as they do when prompted, so I do think it's fair to say that LLMs pass.'
The best-performing AI was GPT-4.5 when it was briefed and told to adopt a persona, followed by Meta's LLaMa-3.1
In 1950, legendary British computer scientist Alan Turing (pictured) proposed the theory of training an AI to give it the intelligence of a child, and then provide the appropriate experiences to build up its intelligence to that of an adult
This is the first time that an AI has passed the test invented by Alan Turing in 1950, according to the new study. The life of this early computer pioneer and the invention of the Turing test was famously dramatised in The Imitation Game, starring Benedict Cumberbatch (pictured)
Last year, another study by the team found two predecessor models from OpenAI – ChatGPT-3.5 and ChatGPT-4 – fooled participants in 50 per cent and 54 per cent of cases (also when told to adopt a human persona).
As GPT-4.5 has now scored 73 per cent, this new suggests that ChatGPT's models are getting better and better at impersonating humans.
It comes 75 years after Alan Turing introduced the ultimate test of computer intelligence in his seminal paper Computing Machinery and Intelligence.
Turing imagined that a human participant would sit at a screen and speak with either a human or a computer through a text-only interface.
If the computer could not be distinguished from a human across a wide range of possible topics, Turing reasoned we would have to admit it was just as intelligent as a human.
A version of the experiment, which asks to you tell the difference between a human and an AI, can be accessed at turingtest.live.
Meanwhile, the pre-print paper is published on online server arXiv and is currently under peer review.
Who was Alan Turing? Pioneering scientist who helped crack Hitler's enigma machine only to be convicted for homosexuality after WWII
Alan Turing (pictured) was a British mathematician best known for his work cracking the enigma code during the Second World War
Alan Turing was a British mathematician born on June 23, 1912 In Maida Vale, London, to father Julius, a civil servant, and mother Ethel, the daughter of a railway engineer.
His talents were recognised early on at school but he struggled with his teachers when he began boarding at Sherborne School aged 13 because he was too fixated on science.
Turing continued to excel at maths but his time at Sherborne was also rocked by the death of his close friend Christopher Morcom from tuberculosis. Morcom was described as Turing's 'first love' and he remained close with his mother following his death, writing to her on Morcom's birthday each year.
He then moved on to Cambridge where he studied at King's College, graduating with a first class degree in mathematics.
During the Second World War, Turing was pivotal in cracking the Enigma codes used by the German military to encrypt their messages.
His work gave Allied leaders vital information about the movement and intentions of Hitler’s forces.
Historians credit the work of Turing and his fellow codebreakers at Bletchley Park in Buckinghamshire with shortening the war by up to two years, saving countless lives, and he was awarded an OBE in 1946 for his services.
Turing is also widely seen as the father of computer science and artificial intelligence due to his groundbreaking work in mathematics in the 1930s.
He was able to prove a 'universal computing machine' would be able to perform equations if they were presented as an algorithm - and had a paper published on the subject in 1936 in the Proceedings of the London Mathematical Society Journal when he was aged just 23.
But he was disgraced in 1952 when he was convicted for homosexual activity, which was illegal at the time and would not be decriminalised until 1967.
To avoid prison, Turing agreed to ‘chemical castration’ – hormonal treatment designed to reduce libido.
As well as physical and emotional damage, his conviction had led to the removal of his security clearance and meant he was no longer able to work for GCHQ, the successor to the Government Code and Cypher School, based at Bletchley Park.
Turing was awarded an OBE in 1946 for his codebreaking work at Bletchley Park, pictured, which is credited with ending World War II two years early
Then In 1954, aged 41, he died of cyanide poisoning. An inquest recorded a verdict of suicide, although his mother and others maintained that his death was accidental.
When his body was discovered, an apple laid half-eaten next to his bed. It was never tested for cyanide but it is speculated it was the source of the fatal dose.
Some more peculiar theories suggest Turing was 'obsessed' with fairytale Snow White and the Seven Dwarfs and his death was inspired by the poisoned apple in the story.
Following a public outcry over his treatment and conviction, the then Prime Minister Gordon Brown issued a public apology in 2009.
He then received a posthumous Royal pardon in 2014, only the fourth to be issued since the end of the Second World War.
It was requested by Justice Secretary Chris Grayling, who described Turing as a national hero who fell foul of the law because of his sexuality.
An e-petition demanding a pardon for Turing had previously received 37,404 signatures.
A 2017 law, that retroactively pardoned all men cautioned or convicted for homosexual acts under historical legislation, was named in his honour.
