Course Information

Welcome to the class website. Here you will find all course related information, and links to all course-related content.

Credit and Acknowledgements

The course content draws extensively from the following:

• “Introduction to Scientific Machine Learning” by Prof. Ilias Bilionis, Dept. of Mechanical Engineering, Purdue University. A big BIG thank you to Prof. Bilionis for making his lecture book and homework assignments available. Please see the original repo at https://predictivesciencelab.github.io/data-analytics-se/index.html.

• “Introduction to Machine Learning” by by Prof. Ryan P. Adams, Dept. of Computer Science, Princeton University, see course web page here. Another big thanks here.

• Educational content available on youtube also by Prof. Ryan Adams

License

This material is available under the GNU General Public License. You may reuse it as you wish, once you include the same License and cite the original repo linked above. You may as well just use the original (Purdue) repo linked above.

Course and Instructor Information

CRNs:

• 60823: Section EN1 - in-person lecture, MW 9-11am, 2045 LUMEB.

• 62822: Section EON - online/asynchronous for remote students.

• 62823: Section ENC - online/asynchronous for on-campus students.

Instructor: Elif Ertekin, Prof. of Mechanical Science and Engineering, University of Illinois, Urbana-Champaign.
email: ertekin [at] illinois [dot] edu
web: https://oasis.mechse.illinois.edu

Course Blurb

This is an introductory but graduate level course for students who are interested in data science/machine learning methods as applied to problems in Engineering and the Physical Sciences. Largely because of my own background in stochastic methods and applied math, we will introduce the concepts from the perspective of probabilistic modeling and highlight the principles behind the methods. Course content will be split approximately evenly between theory and applications.

Topics to be covered:

• intro and review of probability theory (6 lectures - predictive modeling, basics, discrete and continuous random variables, collections of random variables, random vectors)

• uncertainty propagation (4 lectures - sampling, Monte Carlo, Monte Carlo estimators, quantifying uncertainty)

• Bayesian inference (2 lectures - selection of priors, some analytic examples)

• supervised learning (4 lectures - linear and least squares regression, bayesian linear regression, classification)

• unsupervised learning (2 lectures - clustering, density estimation, dimensionality reduction)

• state space models (2 lectures)

• Gaussian process regression (3 lectures - priors, conditioning, Bayesian global optimization)

• neural networks (3 lectures - function approximation, deep, physics-informed)

• sampling methods and variational inference (2 lectures)

Expected outcomes: at the end of the course, you will be able to represent uncertainty in parameters in models, propagate uncertainty through physical models, solve basic supervised tasks (regression, classification, filtering), solve basic unsupervised tasks (clustering, dimensionality reduction, density estimation), design new models that by design include physical information and causal assumptions. You will also gain skills in Python coding, loading and visualizing data in Jupyter notebooks, and use/recognize common Python libraries (pandas, scikit-learn) and software (PyTorch, TensorFlow, JAX).

Course Links

• Online lecturebook (here) : https://elifleaf.github.io/intro-scientific-ml/

• Public access repo : elifleaf/intro-scientific-ml – In-class lecture notes are here

• Slack : tam598scientificml.slack.com

• Canvas (only used for uploading HW pdfs and for recording grades) : https://canvas.illinois.edu/courses/56230

• Lecture recordings : available on TAM 598 Mediaspace Channel

Prerequistes/Suggested Prior Background - What to Expect

• Working knowledge of multivariate calculus and basic linear algebra.

• Basic Python.

• Helpful but optional: knowledge of probability and numerical methods for engineers.

No prior knowledge of machine learning or experience with neural networks is necessary. We plan to build up the necessary tools throughout the semester, with a focus on scientific/physical applications.

If you have no prior experience with Python, go over Lectures 1 to 6 of Prof. Bilionis’s undergraduate course on data science for mechanical engineers. These lectures include a very gentle introduction to the same basic Python concepts, and should take approximately a week to cover.

Office Hours and Course-Related Communication

In-person office hours will be held on Mondays at 11am (after class) in the LUMEB Starbucks.

I can be reached using the course slack at most other times, and will respond to questions as soon as I am able. Slack policy:

• General questions related to course content, homework, etc should be posted and shared with everyone, as this is a small class and many of us will benefit from the discussion.

• If you need to post code or solutions, please post to me privately.

Required Texts and Computer Tools

• This online lecturebook.

• Jupyter notebooks.

Jupyter notebooks are interactive documents that can simultaneously contain text, mathematics, images, and executable code. The executable code can be in many programming languages (e.g., R, Matlab), but we are only going to use Python. We will use Jupyter notebooks for reading activities, hands-on activities, and homework. Understanding the code in Jupyter notebooks requires knowledge of Python, albeit it does not require knowing how to structure and call Python code from the command line. Jupyter notebooks can be run either on your personal computers (instructions vary with operating system and can be found here or in several cloud computing resources. The recommended method for this class is to use Google Colab which is available free of charge and requires only a standard Google account. The activity links included in the course will take you automatically to a copy of the latest version of the corresponding Jupyter notebook which you can then save and edit on your Google Drive.

Recommended Texts

• Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press.

• Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer.

• David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press. Freely available online.

• Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning, Springer. Freely available online.

• Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani, An Introduction to Statistical Learning, Springer. Freely available online.

Grading

Grades are entirely homework based. There will be 7 homework assignments, approximately evenly weighted. Submit each as a single PDF to canvas.

The homework assignments will be both theoretical (e.g., proofs and derivations) and computational (e.g., use a data set to create and fit a model). The assignments will be in the form of a Jupyter notebook with empty space reserved for your writing or coding. The preferred method for inserting equations, proofs, etc is latex. However, if you wish you can do the writing by hand (instead of the latex required by Jupyter notebooks), scan it and submit a single PDF. Submissions should be made through Gradescope.

Grading scale is typical (98-100 A+, 92-98 A, 90-92 A-, …). I may modify the scheme in a way that benefits students.

Disclaimer

This is the first time I have taught this course. Topics and schedule subject to change as I get a sense for timing.