Lectures

Course Introduction and Logistics

Overview of Scientific Machine Learning and Course Structure
▼
Lecture Materials

Course Description and Logistics

Course Motivation

My research on discovering physics from data

Lecture Slides
Homework

Problem Set 0: Linear Algebra and Calculus Recap
Recommended Papers

Scientific discovery in the age of artificial intelligence

Machine learning and big scientific data
Recommended Videos

Physics Informed Machine Learning: High Level Overview of AI and ML in Science and Engineering - Steven Brunton

Science in the Age of Experience 2024 - George Karniadakis, Brown University
Data, Modeling, and Inference

Empirical Laws and Linear Regression
▼
Lecture Materials

A brief history of empirical modeling

Background Lecture Slides

How science will change forever - Coming Soon
Recommended Videos

From Empirical Laws to Linear Regression

An introduction to Dynamics (review) - from data to derivatives
Introduction to Supervised Learning

Linear Regression, Logistic Regression, Feature Engineering, Generalized Linear Models, Maximum Likelihood Estimation, Multiclass Classification
▼
Recommended Readings

CS229 Notes Chapters 1-3, 8, 9
Lecture Materials

Introduction to Supervised Learning Overview Lecture Slides - (see first 10 minutes)

Linear Regression Introduction Notes - Hooke's Law

Code: Linear Regression toy problem (Colab)

Code: Introduction from Linear Regression to Deep Learning - with references

Code Walkthrough Video: Introduction to Scikit-learn

Introduction to Linear Regression

Feature Engineering and Generalization

Introduction to Logistic Regression - Coming Soon

Why Ordinary Least Squares? Maximum Likelihood Estimation
Recommended Videos

CS229 Introduction to Machine Learning (Lecture 1)

CS229 Linear Regression and Gradient Descent (Lecture 2)

CS229 Locally Weighted and Logistic Regression (Lecture 3)

CS229 Perceptron & Generalized Linear Model (Lecture 4)

CS229 Data Splits, Models and Cross-Validation (Lecture 8)
Introduction to Deep Learning

Nonlinear Predictors, Neural Networks, Activation Functions, Backpropagation, etc
▼
Recommended Readings

CS229 Notes Chapter 7
Lecture Materials

Introduction to Deep Learning - Lecture Slides

Code: Deep Learning Tutorial

Introduction to Deep Learning Video
Recommended Videos

CS229 Introduction to Neural Networks

Learning Deep Learning with GPT

Neural Networks 3Blue1Brown Series
Time Series Analysis

Time series analysis, autoregression, recurrent neural networks, state-space models, probabilistic models, etc.
▼
Suggested Motivation Videos

What is Time Series Analysis?

Time Series Forecasting with Machine Learning
Lecture Materials

Time series analysis lecture notes
Modeling the World with Differential Equations

ODEs, PDEs, Complexity and Uncertainty
▼
Lecture Materials

Pondering on the Nature of Complexity in Science and Machine Learning

Everything is a Function

Complex Systems and Differential Equations

Introduction to Numerical Integration

The story of the Lorenz System: Nonlinearity, Chaos, Uncertainty Quantification

Introduction to Partial Differential Equations
Recommended Videos

Differential equations, a tourist's guide - 3Blue1Brown

But what is a partial differential equation? - 3Blue1Brown
Symbolic Identification of Differential Equations from Data

Given data in space and time, how can we find its corresponding equation using sparse linear regression?
▼
Recommended Readings

Discovering governing equations from data by sparseidentification of nonlinear dynamical systems

Data-driven discovery of partial differential equations

Distilling Free-Form Natural Laws from Experimental Data
Recommended Videos

Sparse Identification of Nonlinear Dynamics (SINDy)

Python Symbolic Regression (PySR)

ETH Zürich AISE: Symbolic Regression and Model Discovery
Lecture Materials

Sprase Identification of Differential Equations Intro

Symbolic Regression with Genetic Algorithms
Residual Minimization & Physics Informed Neural Networks

Including physical knowledge in a loss function
▼
Recommended Readings

Physics Informed Neural Networks

So, what is a physics-informed neural network? - Ben Moseley
Recommended Videos

Intro to Physics Informed Neural Networks

Lagrangian Neural Networks
Lecture Materials

Intro to PINNs Lecture Notes

Simple Example Code
Unsupservised Learning

How to deal with features without ground truth? It's a much harder problem.
▼
Recommended Readings

K-Means Clustering - CS229 Chapter 10

Gaussian Mixture Model - CS229 Chapter 11

Principal Component Analysis - CS229 Chapter 12
Recommended Videos

Latent Space Visualisation: PCA, t-SNE, UMAP

Gaussian Mixture Models
Lecture Materials

K-Means Intro Slides

GMM Intro Slides

PCA Intro Slides

Explanation with Code: SVD with Demo

Other unsupervised learning algorithms
Modal Decomposition in Spatio-Temporal Systems

When the dynamics are high dimensional, dimensionality reduction is a useful trick
▼
Recommended Readings

Reduced-Order Models (ROMs) - Data-Driven Science and Engineering Chapter 12

Dynamic Mode Decomposition - Data-Driven Science and Engineering Section 7.2

Dynamic mode decomposition of numerical and experimental data

Data-driven operator inference for nonintrusive projection-based model reduction
Recommended Videos

Introduction to Dynamic Mode Decomposition

Proper Orthogonal Decomposition for Partial Differential Equations (Part 1)

Proper Orthogonal Decomposition for Partial Differential Equations (Part 2)

Introduction to Reduced Order Modeling

An Introduction to the Koopman Operator (Series) - Optional
Lecture Material

Code: DMD for flow around a cylinder

Code: POD/SVD for flow around a cylinder