Progress in machine classification of images The error rate of AI by year. Red line - the error rate of a trained human on a particular task.
Progress in artificial intelligence (AI) refers to the advances, milestones, and breakthroughs that have been achieved in the field of artificial intelligence over time. AI is a multidisciplinary branch of computer science that aims to create machines and systems capable of performing tasks that typically require human intelligence. Artificial intelligence applications have been used in a wide range of fields including medical diagnosis, economic-financial applications, robot control, law, scientific discovery, video games, and toys. However, many AI applications are not perceived as AI: "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore."[1][2] "Many thousands of AI applications are deeply embedded in the infrastructure of every industry."[3] In the late 1990s and early 21st century, AI technology became widely used as elements of larger systems,[3][4] but the field was rarely credited for these successes at the time.
Kaplan and Haenlein structure artificial intelligence along three evolutionary stages: 1) artificial narrow intelligence – applying AI only to specific tasks; 2) artificial general intelligence – applying AI to several areas and able to autonomously solve problems they were never even designed for; and 3) artificial super intelligence – applying AI to any area capable of scientific creativity, social skills, and general wisdom.[2]
To allow comparison with human performance, artificial intelligence can be evaluated on constrained and well-defined problems. Such tests have been termed subject matter expert Turing tests. Also, smaller problems provide more achievable goals and there are an ever-increasing number of positive results.
Humans still substantially outperform both GPT-4 and models trained on the ConceptARC benchmark that scored 60% on most, and 77% on one category, while humans 91% on all and 97% on one category.[5]
There are many useful abilities that can be described as showing some form of intelligence. This gives better insight into the comparative success of artificial intelligence in different areas.
AI, like electricity or the steam engine, is a general-purpose technology. There is no consensus on how to characterize which tasks AI tends to excel at.[15] Some versions of Moravec's paradox observe that humans are more likely to outperform machines in areas such as physical dexterity that have been the direct target of natural selection.[16] While projects such as AlphaZero have succeeded in generating their own knowledge from scratch, many other machine learning projects require large training datasets.[17][18] Researcher Andrew Ng has suggested, as a "highly imperfect rule of thumb", that "almost anything a typical human can do with less than one second of mental thought, we can probably now or in the near future automate using AI."[19]
Games provide a high-profile benchmark for assessing rates of progress; many games have a large professional player base and a well-established competitive rating system. AlphaGo brought the era of classical board-game benchmarks to a close when Artificial Intelligence proved their competitive edge over humans in 2016. Deep Mind's AlphaGo AI software program defeated the world's best professional Go Player Lee Sedol.[20] Games of imperfect knowledge provide new challenges to AI in the area of game theory; the most prominent milestone in this area was brought to a close by Libratus' poker victory in 2017.[21][22]E-sports continue to provide additional benchmarks; Facebook AI, Deepmind, and others have engaged with the popular StarCraft franchise of videogames.[23][24]
Broad classes of outcome for an AI test may be given as:
optimal: it is not possible to perform better (note: some of these entries were solved by humans)
Heads-up limit hold'em poker: Statistically optimal in the sense that "a human lifetime of play is not sufficient to establish with statistical significance that the strategy is not an exact solution" (2015)[27]
Proposed "universal intelligence" tests aim to compare how well machines, humans, and even non-human animals perform on problem sets that are generic as possible. At an extreme, the test suite can contain every possible problem, weighted by Kolmogorov complexity; however, these problem sets tend to be dominated by impoverished pattern-matching exercises where a tuned AI can easily exceed human performance levels.[70][71][72][73][74]
Exams
According to OpenAI, in 2023 ChatGPTGPT-4 scored the 90th percentile on the Uniform Bar Exam. On the SATs, GPT-4 scored the 89th percentile on math, and the 93rd percentile in Reading & Writing. On the GREs, it scored on the 54th percentile on the writing test, 88th percentile on the quantitative section, and 99th percentile on the verbal section. It scored in the 99th to 100th percentile on the 2020 USA Biology Olympiad semifinal exam. It scored a perfect "5" on several AP exams.[75]
Independent researchers found in 2023 that ChatGPT GPT-3.5 "performed at or near the passing threshold" for the three parts of the United States Medical Licensing Examination. GPT-3.5 was also assessed to attain a low, but passing, grade from exams for four law school courses at the University of Minnesota.[75] GPT-4 passed a text-based radiology board–style examination.[76][77]
Many competitions and prizes, such as the Imagenet Challenge, promote research in artificial intelligence. The most common areas of competition include general machine intelligence, conversational behavior, data-mining, robotic cars, and robot soccer as well as conventional games.[78]
Past and current predictions
An expert poll around 2016, conducted by Katja Grace of the Future of Humanity Institute and associates, gave median estimates of 3 years for championship Angry Birds, 4 years for the World Series of Poker, and 6 years for StarCraft. On more subjective tasks, the poll gave 6 years for folding laundry as well as an average human worker, 7–10 years for expertly answering 'easily Googleable' questions, 8 years for average speech transcription, 9 years for average telephone banking, and 11 years for expert songwriting, but over 30 years for writing a New York Times bestseller or winning the Putnam math competition.[79][80][81]
AlphaGo defeated a European Go champion in October 2015, and Lee Sedol in March 2016, one of the world's top players (see AlphaGo versus Lee Sedol). According to Scientific American and other sources, most observers had expected superhuman Computer Go performance to be at least a decade away.[85][86][87]
Estimates when computers would exceed humans at Go
AI pioneer and economist Herbert A. Simon inaccurately predicted in 1965: "Machines will be capable, within twenty years, of doing any work a man can do". Similarly, in 1970 Marvin Minsky wrote that "Within a generation... the problem of creating artificial intelligence will substantially be solved."[93]
Four polls conducted in 2012 and 2013 suggested that the median estimate among experts for when AGI would arrive was 2040 to 2050, depending on the poll.[94][95]
The Grace poll around 2016 found results varied depending on how the question was framed. Respondents asked to estimate "when unaided machines can accomplish every task better and more cheaply than human workers" gave an aggregated median answer of 45 years and a 10% chance of it occurring within 9 years. Other respondents asked to estimate "when all occupations are fully automatable. That is, when for any occupation, machines could be built to carry out the task better and more cheaply than human workers" estimated a median of 122 years and a 10% probability of 20 years. The median response for when "AI researcher" could be fully automated was around 90 years. No link was found between seniority and optimism, but Asian researchers were much more optimistic than North American researchers on average; Asians predicted 30 years on average for "accomplish every task", compared with the 74 years predicted by North Americans.[79][80][81]
Estimates of when AGI will arrive
Year prediction made
Predicted year
Number of years
Predictor
Contemporaneous source
1965
1985 or sooner
20 or less
Herbert A. Simon
The shape of automation for men and management[93][96]
^ abKaplan, Andreas; Haenlein, Michael (2019). "Siri, Siri, in my hand: Who's the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence". Business Horizons. 62: 15–25. doi:10.1016/j.bushor.2018.08.004. S2CID158433736.
^Ontanon, Santiago; Synnaeve, Gabriel; Uriarte, Alberto; Richoux, Florian; Churchill, David; Preuss, Mike (December 2013). "A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft". IEEE Transactions on Computational Intelligence and AI in Games. 5 (4): 293–311. CiteSeerX10.1.1.406.2524. doi:10.1109/TCIAIG.2013.2286295. S2CID5014732.
^Tesauro, Gerald (January 2002). "Programming backgammon using self-teaching neural nets". Artificial Intelligence. 134 (1–2): 181–199. doi:10.1016/S0004-3702(01)00110-2. ...at least two other neural net programs also appear to be capable of superhuman play
^Nickel, Arno (May 2017). "Zor Winner in an Exciting Photo Finish". www.infinitychess.com. Innovative Solutions. Archived from the original on 2018-08-17. Retrieved 2018-07-17. ... on third place the best centaur ...
^Roeder, Oliver (10 July 2017). "The Bots Beat Us. Now What?". FiveThirtyEight. Archived from the original on 28 December 2017. Retrieved 28 December 2017.
^Proverb: The probabilistic cruciverbalist. By Greg A. Keim, Noam Shazeer, Michael L. Littman, Sushant Agarwal, Catherine M. Cheves, Joseph Fitzgerald, Jason Grosland, Fan Jiang, Shannon Pollard, and Karl Weinmeister. 1999. In Proceedings of the Sixteenth National Conference on Artificial Intelligence, 710-717. Menlo Park, Calif.: AAAI Press.
^Santoro, Adam; Bartunov, Sergey; Botvinick, Matthew; Wierstra, Daan; Lillicrap, Timothy (19 May 2016). "One-shot Learning with Memory-Augmented Neural Networks". p. 5, Table 1. arXiv:1605.06065 [cs.LG]. 4.2. Omniglot Classification: "The network exhibited high classification accuracy on just the second presentation of a sample from a class within an episode (82.8%), reaching up to 94.9% accuracy by the fifth instance and 98.1% accuracy by the tenth.
^ abcZhang, D., Mishra, S., Brynjolfsson, E., Etchemendy, J., Ganguli, D., Grosz, B., ... & Perrault, R. (2021). The AI index 2021 annual report. AI Index (Stanford University). arXiv preprint arXiv:2103.06312.
^Brynjolfsson, E., & Mitchell, T. (2017). What can machine learning do? Workforce implications. Science, 358(6370), 1530-1534.
^Nie, W., Yu, Z., Mao, L., Patel, A. B., Zhu, Y., & Anandkumar, A. (2020). Bongard-logo: A new benchmark for human-level concept learning and reasoning. Advances in Neural Information Processing Systems, 33, 16468-16480.
^Stephenson, Matthew; Renz, Jochen; Ge, Xiaoyu (March 2020). "The computational complexity of Angry Birds". Artificial Intelligence. 280: 103232. arXiv:1812.07793. doi:10.1016/j.artint.2019.103232. S2CID56475869. Despite many different attempts over the past five years the problem is still largely unsolved, with AI approaches far from human-level performance.
^Schoenick, Carissa; Clark, Peter; Tafjord, Oyvind; Turney, Peter; Etzioni, Oren (23 August 2017). "Moving beyond the Turing Test with the Allen AI Science Challenge". Communications of the ACM. 60 (9): 60–64. arXiv:1604.04315. doi:10.1145/3122814. S2CID6383047.
^Feigenbaum, Edward A. (2003). "Some challenges and grand challenges for computational intelligence". Journal of the ACM. 50 (1): 32–40. doi:10.1145/602382.602400. S2CID15379263.
^Hernandez-Orallo, Jose (2000). "Beyond the Turing Test". Journal of Logic, Language and Information. 9 (4): 447–466. doi:10.1023/A:1008367325700. S2CID14481982.
^ abGrace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2017). When will AI exceed human performance? Evidence from AI experts. arXiv preprint arXiv:1705.08807.
^"Cracking GO". IEEE Spectrum: Technology, Engineering, and Science News. 2007. Archived from the original on 31 January 2018. Retrieved 31 January 2018.
^Khatchadourian, Raffi (16 November 2015). "The Doomsday Invention". The New Yorker. Archived from the original on 29 April 2019. Retrieved 31 January 2018.
^Müller, V. C., & Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. In Fundamental issues of artificial intelligence (pp. 555-572). Springer, Cham.
^Muehlhauser, L., & Salamon, A. (2012). Intelligence explosion: Evidence and import. In Singularity Hypotheses (pp. 15-42). Springer, Berlin, Heidelberg.
_artificial_intelligence_icon.png){kind=icon}
