Lecture 3 - Building Frontier AI
| ← Back to AICS | Next: Lecture 4 → |
Lecture 3 discusses how modern frontier AI systems are built. We discuss scaling laws, memory systems (including context/RAG), large reasoning models, and agentic AI. We discuss the capabilities of modern systems as measured by benchmark tests, and unpack what may still be missing for artificial general intelligence - including consolidation, metacognition, and embodiment.
What you need to understand:
- Sutton’s “bitter lesson” for AI development
- Retrieval augmented generation
- Chain of thought reasoning
- Tool use and scaffolds for AI agents
- The Jagged Frontier
Sample essay questions
Do modern frontier AI systems have ‘general intelligence’? If not, what is missing?
In his essay “The Bitter Lesson” the AI pioneer Rich Sutton argues that general-purpose computation and big data are the key ingredients of intelligence. Do you agree?