"You can't stop the signal, Mal. Everything goes somewhere, and I go everywhere."
-- Mr. Universe, from the movie "Serenity"
"If you torture data sufficiently, it will confess to almost anything."
-- Fred Menger

Course Description

This course is part of the University of Idaho series on Artificial Intelligence and Machine Learning which comprises these three courses: Artificial Intelligence, Machine Learning, and Evolutionary Computation. The department also offers related courses: CS474/574 Deep Learning and CS489 Semantic Web.

Prereq: Solid C/C++ programming skills - no... really, solid understanding of basic linear algebra, calculus, probability/statistics, UNIX skills desirable, CS210 (because many algorithms are explained using multiple programming languages) or instructor permission.

We will focus on the machine learning half of Artificial Intelligence. This includes algorithms to learn behavior, classify data, and predict behaviors and/or data given past data. This is an implementation course. We will implement some of the important algorithms of machine learning and apply them to small problems (usually under 1000 samples of data). This is a course about the algorithms and not about using pre-made tools to do machine learning and data mining. We will look at topics such as decision trees, neural networks, deep learning, Markov brains, and reinforcement learning. There will be lots of programming in C/C++ and algorithm work. Proficiency in programming in C/C++ is a must as well as a solid understanding of undergrad math including an understanding of basic probability and statistics, probability distributions, calculus, and linear algebra. Algorithm descriptions will be given in Python and demonstrations in Mathematica (purchase not required).

Also available at the University of Idaho is CS477 which is how to use machine learning tools and Python toolkits to solve machine learning problems. The Math department also offers a course in the mathematics behind machine learning.

Hopefully this will be fun and entertaining.

Time: MWF 9:30-10:30 Pacific Time
Final: None

Location: EP 204 Masks are required in class, vaccines highly recommended
Textbook: Machine Learning: An Algorithmic Perspective by Stephen Marsland (Second Edition!)
Optional Ref: The Quick Python Book 3nd Ed. by Naomi Ceder
Really great and easy to read book. Fine index.

Estimated Syllabus

This syllabus is AN ESTIMATE of what we might cover this semester. The class varies from semester to semester to reflect new and interesting topics.

Week

Monday
of that
Week

 Topics/Links Assignments Comments
wk 1 Jan 10 Artificial Intelligence, Machine Learning, Data Analytics the big picture, Learning vs Prediction, The classes of problems Read Chapter 1 NO CLASS MONDAY
wk 2 Jan 17 Examining and cleaning the input, missing data, scaling, curse of dimensionality Read chapter 2 NO CLASS MONDAY
wk 3 Jan 24 The Matrix Library, Simple perceptron, single layer neural networks, matrix version of single layer algorithm, sigmoid functions, demonstration of algorithms running Read chapter 3, Assignment 0  
wk 4 Jan 31 training, validation, testing, measures of quality, ROC space, recursive networks, evolutionary training of neural networks Assignment 1  
wk 5 Feb 7 Multilayer NN and backprop, convergence behavior and demo. Time series and deep neural networks Assignment 2, Read chapter 2.4  
wk 6 Feb 14 Dimensionality reduction and PCAs. Handout on the PCA algorithm that might be useful. Read chapter 6.2 on  
wk 7 Feb 21 PCA and Linear Discriminant Analysis (LDA) Read chapter 6, Assignment 3 NO CLASS MONDAY
wk 8 Feb 28 Dimensionality reduction and PCAs. Handout on the PCA algorithm that might be useful. Read chapter 6.1, chapter 7  
wk 9 Mar 7 Nearest Neighbor Methods and KDTrees Read chapter 7, Assignment 4  
wk 10 Mar 14   SPRING BREAK! NO CLASS THIS WEEK
wk 11 Mar 21 K-Means,EM Read Chapter 7.2 Assignment 5  
wk 12 Mar 28 EM, Decision Trees Read Chapters 14.1 and 12 Assignment 6  
wk 13 Apr 4 Entropy/Information, Decision Trees, Random Forests Read chapter 13, Assignment 7  
wk 13 Apr 11 Ensemble Learning    
wk 14 Apr 18 Reinforcement Learning, Q-Learning, Sarsa, eligibility traces    
wk 15 Apr 25 eligibility traces, Q-learning in games   NO CLASS THURSDAY
wk 16 May 2 Market Basket Analysis, Stochastic Learning with the EM algorithm or Text analysis    
wk 17 May 9 Finals Week Final: None  

References and Resources

Some Videos

Services

  • The class has its own machine with more up-to-date compilers than maybe in other classes! The machine is: cs-475.cs.uidaho.edu. Give it a try and report any problems to cshelp@uidaho.edu if there is a problem. The machine is only visible on campus. If you are outside the university firewall you will need to VPN onto campus. To be allowed to VPN, you need to sign up with IT by contacting helpdesk@uidaho.edu to have your student account added to the VPN group.
  • CSAC is the Computer Science Assistance Center and is staffed with people to help with things like simple programming questions or problems with logging onto a CS machine. They are now located in JEB 211.
  • Homeworks for CS475 are submitted for testing using this secure submission page. The last submission before the due date will be graded. This is the only place to submit homeworks.
  • How Testing Works

Policies and Processes

Fun Stuff