Intro
Preface
Contents
Acronyms
Author's Note
Part I Foundations
1 AI Sculpture
1.1 Manifolds in High Dimensions
1.2 Sculpting Process
1.3 Notational Convention
1.4 Regression and Classification
1.4.1 Linear Regression and Logistic Regression
1.4.2 Regression Loss and Cross-Entropy Loss
1.4.3 Sculpting with Shades
1.5 Discriminative and Generative AI
1.6 Success of Discriminative Methods
1.7 Feature Engineering in Classical ML
1.8 Supervised and Unsupervised AI
1.9 Beyond Manifolds
1.10 Chapter Summary
2 Make Me Learn

2.1 Learnable Parameters
2.1.1 The Power of a Single Neuron
2.1.2 Neurons Working Together
2.2 Backpropagation of Gradients
2.2.1 Partial Derivatives
2.2.2 Forward and Backward Passes
2.3 Stochastic Gradient Descent
2.3.1 Handling Difficult Landscapes
2.3.2 Stabilization of Training
2.4 Chapter Summary
3 Images and Sequences
3.1 Convolutional Neural Networks
3.1.1 The Biology of the Visual Cortex
3.1.2 Pattern Matching
3.1.3 3-D Convolution
3.2 Recurrent Neural Networks
3.2.1 Neurons with States
3.2.2 The Power of Recurrence

3.2.3 Going Both Ways
3.2.4 Attention
3.3 Self-Attention
3.4 LSTM
3.5 Beyond Images and Sequences
3.6 Chapter Summary
4 Why AI Works
4.1 Convex Polytopes
4.2 Piecewise Linear Function
4.2.1 Subdivision of the Input Space
4.2.2 Piecewise Non-linear Function
4.2.3 Carving Out the Feature Spaces
4.3 Expressive Power of AI
4.4 Convolutional Neural Network
4.5 Recurrent Neural Network
4.6 Architectural Variations
4.7 Attention and Carving
4.8 Optimization Landscape
4.8.1 Graph-Induced Polynomial
4.8.2 Gradient of the Loss Function

4.8.3 Visualization
4.8.4 Critical Points
4.9 The Mathematics of Loss Landscapes
4.9.1 Random Polynomial Perspective
4.9.2 Random Matrix Perspective
4.9.3 Spin Glass Perspective
4.9.4 Computational Complexity Perspective
4.9.5 SGD and Critical Points
4.9.6 Confluence of Perspectives
4.10 Distributed Representation and Intrinsic Dimension
4.11 Chapter Summary
5 Novice to Maestro
5.1 How AI Learns to Sculpt
5.1.1 Training Data
5.1.2 Evaluation Metrics
5.1.3 Hyperparameter Search
5.1.4 Regularization
5.1.5 Bias and Variance

5.1.6 A Fairy Tale in the Land of ML
5.2 Learning Curves
5.3 From the Lab to the Dirty Field
5.4 System Design
5.5 Flavors of Learning
5.6 Ingenuity and Big Data in the Success of AI
5.7 Chapter Summary
6 Unleashing the Power of Generation
6.1 Creating Universes
6.2 To Recognize It, Learn to Draw It
6.3 General Definition
6.4 Generative Parameters
6.5 Generative AI Models
6.5.1 Restricted Boltzmann Machines
6.5.2 Autoencoders
6.5.3 Variational Autoencoder
6.5.4 Pixel Recursive Models
6.5.5 Generative Adversarial Networks