The Definitive History of AI: Part 5 - The Second AI Winter and the Rise of Machine Learning (1987-2000)
Welcome to the fifth installment of our seven-part series on the history of Artificial Intelligence. Today, we'll explore the period from 1987 to 2000, which encompasses the Second AI Winter and the subsequent rise of machine learning approaches.
The Onset of the Second AI Winter
The enthusiasm and investment that characterized the expert systems boom began to wane in the late 1980s, leading to what is known as the Second AI Winter. Several factors contributed to this downturn:
- Collapse of the AI Hardware Market: Specialized AI hardware, particularly LISP machines, became obsolete as general-purpose computers became more powerful and cost-effective.
- Limitations of Expert Systems: The brittleness and maintenance challenges of expert systems became increasingly apparent, limiting their practical applications.
- Unfulfilled Promises: Many of the ambitious goals set during the AI boom, such as fully functional natural language processing, remained elusive.
- Funding Cuts: Government agencies, particularly in the US and UK, significantly reduced AI research funding.
The Strategic Computing Initiative
The U.S. Strategic Computing Initiative, launched in 1983, was a significant government effort to fund AI research. However, by the late 1980s, the program faced criticism for its lack of tangible results. In 1987, the program was restructured to focus more on specific military applications rather than general AI capabilities. This shift marked a broader trend of moving away from ambitious, general AI goals towards more practical, focused applications.
Paradigm Shift: From Knowledge-Based to Data-Driven Approaches
As the limitations of traditional AI approaches became clear, researchers began to explore alternative methods. This period saw a gradual shift from knowledge-based systems (like expert systems) to data-driven approaches, setting the stage for the machine learning revolution:
- Probabilistic Reasoning: Techniques like Bayesian networks gained prominence, allowing AI systems to handle uncertainty more effectively.
- Machine Learning: Algorithms that could learn from data, rather than relying on hand-coded rules, began to show promise.
- Neural Networks Renaissance: After years of being largely ignored, neural networks experienced a resurgence of interest, particularly with the development of backpropagation algorithms.
Key Developments in Machine Learning
Despite the overall reduction in AI funding and interest, this period saw several crucial developments in machine learning:
- Backpropagation (1986): While discovered earlier, the importance of backpropagation for training neural networks became widely recognized, thanks to the work of researchers like Geoffrey Hinton.
- Support Vector Machines (1995): Developed by Vladimir Vapnik and Corinna Cortes, SVMs became a powerful tool for classification and regression tasks.
- Random Forests (1995): This ensemble learning method, introduced by Tin Kam Ho, combined multiple decision trees to improve prediction accuracy.
- Long Short-Term Memory (LSTM) Networks (1997): Developed by Sepp Hochreiter and Jürgen Schmidhuber, LSTMs addressed the vanishing gradient problem in recurrent neural networks, enabling better processing of sequential data.
AI in Academia and Industry
During this period, AI research and development continued, albeit with less hype and more focused goals:
- Academic Research: Universities continued to conduct AI research, often with a greater emphasis on theoretical foundations and machine learning.
- Industrial Applications: Companies began to adopt machine learning techniques for specific applications, such as fraud detection in financial services and recommendation systems in e-commerce.
- Data Mining: The growing availability of digital data led to increased interest in techniques for extracting useful patterns and knowledge from large datasets.
Landmark AI Achievements
Despite the challenges, this period saw some notable AI achievements:
- Deep Blue (1997): IBM's chess-playing computer defeated world champion Garry Kasparov, marking a significant milestone in game-playing AI.
- NASA's Remote Agent (1999): This AI system became the first onboard autonomous planning and control system to control a spacecraft.
- DARPA's CALO Project (started in 2003): While beginning just after our current period, this project laid the groundwork for personal assistant AIs like Siri.
The Internet and the Growth of Data
The rapid growth of the internet during the 1990s had profound implications for AI:
- It led to an explosion in the amount of available digital data, which would prove crucial for training machine learning models.
- It provided a platform for distributing and accessing AI technologies and applications.
- It created new problem domains for AI, such as web search and online recommendation systems.
Theoretical Advances
This period also saw important theoretical advances that would underpin future AI developments:
- Computational Learning Theory: Work by researchers like Leslie Valiant provided a theoretical framework for understanding machine learning.
- Reinforcement Learning: The publication of Richard Sutton and Andrew Barto's book "Reinforcement Learning: An Introduction" in 1998 helped establish this as a key area of AI research.
- Probabilistic Graphical Models: Work by researchers like Judea Pearl advanced our understanding of reasoning under uncertainty.
Conclusion
The period from 1987 to 2000 was a time of transition for AI. The Second AI Winter saw a reduction in funding and a tempering of expectations, but it also drove a shift towards more practical, data-driven approaches. The rise of machine learning during this time laid the foundation for the AI boom that would follow in the 21st century. This era demonstrated the resilience of the AI field, showing how setbacks and challenges could lead to new paradigms and approaches that would ultimately advance the science of artificial intelligence.