The Review Of The Previous Class
Stateaction Rewards
Finite Horizon MDPS
The Concept Of Dynamical Systems
Examples Of Dynamical Models
Linear Quadratic Regulation (LQR)
Linearizing A Nonlinear Model
Computing Rewards
Riccati Equation
Ng, Andrew
Ng's research is in the areas of machine learning and artificial intelligence. He leads the STAIR (STanford Artificial Intelligence Robot) project, whose goal is to develop a home assistant robot that can perform tasks such as tidy up a room, load/unload a dishwasher, fetch and deliver items, and prepare meals using a kitchen. Since its birth in 1956, the AI dream has been to build systems that exhibit "broad spectrum" intelligence. However, AI has since splintered into many different subfields, such as machine learning, vision, navigation, reasoning, planning, and natural language processing. To realize its vision of a home assistant robot, STAIR will unify into a single platform tools drawn from all of these AI subfields. This is in distinct contrast to the 30yearold trend of working on fragmented AI subfields, so that STAIR is also a unique vehicle for driving forward research towards true, integrated AI.
Ng also works on machine learning algorithms for robotic control, in which rather than relying on months of human handengineering to design a controller, a robot instead learns automatically how best to control itself. Using this approach, Ng's group has developed by far the most advanced autonomous helicopter controller, that is capable of flying spectacular aerobatic maneuvers that even experienced human pilots often find extremely difficult to execute. As part of this work, Ng's group also developed algorithms that can take a single image,and turn the picture into a 3D model that one can flythrough and see from different angles.
info.pdf  Course Information 
schedule.pdf  Course Schedule 
AIclasses.pdf  Other AI Courses 
cs229notes1.pdf  Linear Regression, Classification and logistic regression, Generalized Linear Models 
cs229notes2.pdf  Generative Learning algorithms 
cs229notes3.pdf  Support Vector Machines 
cs229notes4.pdf  Learning Theory 
cs229notes5.pdf  Regularization and model selection 
cs229notes6.pdf  The perceptron and large margin classifiers 
cs229notes7a.pdf  The kmeans clustering algorithm 
cs229notes7b.pdf  Mixtures of Gaussians and the EM algorithm 
cs229notes8.pdf  The EM algorithm 
cs229notes9.pdf  Factor analysis 
cs229notes10.pdf  Principal components analysis 
cs229notes11.pdf  Independent Components Analysis 
cs229notes12.pdf  Reinforcement Learning and Control 
Linear Algebra Review and Reference  cs229linalg.pdf  
Probability Theory Review  cs229prob.pdf  
Matlab Review 


Convex Optimization Overview, Part I  cs229cvxopt.pdf  
Convex Optimization Overview, Part II  cs229cvxopt2.pdf  
Hidden Markov Models  cs229hmm.pdf  
Gaussian Processes 

Advice on applying machine learning: Slides from Andrew's lecture on getting machine learning algorithms to work in practice can be found here. 
Previous projects: A list of last year's final projects can be found here. 
Matlab Resources Here are a couple of Matlab tutorials that you might find helpful: Matlab Tutorial and A Practical Introduction to Matlab. For emacs users only: If you plan to run Matlab in emacs, here are matlab.el, and a helpful emac's file. 
Octave Resources For a free alternative to Matlab, check out GNU Octave. The official documentation is available here. Some useful tutorials on Octave include Octave Tutorial and Octave on Wiki. 
Viewing PostScript and PDF files: Depending on the computer you are using, you may be able to download a PostScript viewer or PDF viewer for it if you don't already have one. 
Assignment  Assignment Data Files  Solution  Solution Data Files 

Problem Set 1  PS1data.zip  Solution Set 1  ps1_solutiondata.zip 
Problem Set 2  PS2data.zip  Solution Set 2  
Problem Set 3  PS3data.zip  Solution Set 3  ps3_solutiondata.zip 
Problem Set 4  PS4data.zip  Solution Set 4  ps4_solutiondata.zip 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 9 min  
Topics: The Motivation & Applications of Machine Learning, The Logistics of the Class, The Definition of Machine Learning, The Overview of Supervised Learning, The Overview of Learning Theory, The Overview of Unsupervised Learning, The Overview of Reinforcement Learning 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 16 min  
Topics: An Application of Supervised Learning  Autonomous Deriving, ALVINN, Linear Regression, Gradient Descent, Batch Gradient Descent, Stochastic Gradient Descent (Incremental Descent), Matrix Derivative Notation for Deriving Normal Equations, Derivation of Normal Equations 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 13 min  
Topics: The Concept of Underfitting and Overfitting, The Concept of Parametric Algorithms and Nonparametric Algorithms, Locally Weighted Regression, The Probabilistic Interpretation of Linear Regression, The motivation of Logistic Regression, Logistic Regression, Perceptron 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 14 min  
Topics: Multinomial Event Model, Nonlinear Classifiers, Neural Network, Applications of Neural Network, Intuitions about Support Vector Machine (SVM), Notation for SVM, Functional and Geometric Margins 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 14 min  
Topics: Bias/variance Tradeoff, Empirical Risk Minimization (ERM), The Union Bound, Hoeffding Inequality, Uniform Convergence  The Case of Finite H, Sample Complexity Bound, Error Bound, Uniform Convergence Theorem & Corollary 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 22 min  
Topics: Bayesian Statistics and Regularization, Online Learning, Advice for Applying Machine Learning Algorithms, Debugging/fixing Learning Algorithms, Diagnostics for Bias & Variance, Optimization Algorithm Diagnostics, Diagnostic Example  Autonomous Helicopter, Error Analysis, Getting Started on a Learning Problem 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 17 min  
Topics: Latent Semantic Indexing (LSI), Singular Value Decomposition (SVD) Implementation, Independent Component Analysis (ICA), The Application of ICA, Cumulative Distribution Function (CDF), ICA Algorithm, The Applications of ICA 
Watch Online: 
Download:
Right Click, and Save As

Duration:  
Watch Now  Download  1 hr 16 min  
Topics: Advice for Applying Machine Learning, Debugging Reinforcement Learning (RL) Algorithm, Linear Quadratic Regularization (LQR), Differential Dynamic Programming (DDP), Kalman Filter & Linear Quadratic Gaussian (LQG), Predict/update Steps of Kalman Filter, Linear Quadratic Gaussian (LQG) 