Schedule and Assignments

The following is a tentative schedule of topics that will be considered in our weekly meetings. In each weekly assignment, I will itemize the readings that are required and will frequently suggest optional readings.

Week of 9/12 Introduction to Machine Learning; Linear Models and Perceptrons

Assignment writeup
Readings
Chapter 1: Introduction
Chapter 2: Supervised Learning
Chapter 10: Linear Discrimination, Sections 10.1, 10.3, and 10.4 only
Chapter 11: Perceptrons, Sections 11.1-11.4 only
Fawcett and Provost, "Combining Data Mining and Machine Learning for Effective User Profiling", Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96). AAAI Press, 1996.

Week of 9/19 Probabilistic Models: Naive Bayes

Assignment writeup
Readings
Chapter 3: Bayesian Decision Theory (up to page 55)
Mitchell: Section 3.4 2, page 59 only
Witten and Frank: Section 4.2
Duda, Hart, and Stork: Section 2.1

Week of 9/26 Regression

Assignment writeup
Readings
Sections 4.1, 4.2, 4.5
Sections 4.6 and 5.8
Frank et al, "Naive Bayes for Regression", Machine Learning, Vol. 41, No. 1, pages 5-25.

Week of 10/3 Decision Trees

Assignment writeup
Readings
Sections 9.1-9.3
Mitchell: Chapter 3
Russell and Norvig: Section 18.3
Quinlan, "Induction of Decision Trees", Machine Learning, Vol. 1, No. 1., 1986.
Page and Ray "Skewing: An Efficient Alternative to Lookahead for Decision Tree Induction", IJCAI-03

Week of 10/10 K-Nearest Neighbor (Short topic, due to Fall Reading Period)

Assignment writeup
Readings
Mitchell: pages 230-236
Russell and Norvig: pages 733-735
Witten and Frank: pages 78-79 and 128-135
Mitchell: pages 236-238
"Locally Weighted Regression", Artificial Intelligence Review, 1997.

Week of 10/17 Neural Networks

Assignment writeup
Readings
Mitchell: Chapter 4 (at least through Section 4.6.2)
Witten and Frank: pages 227-233
"Multitask Learning", Machine Learning, 1997.

Week of 10/24 Support Vector Machines

Assignment writeup
Required Readings
Alpaydin, Sections 10.2 and 10.9
Alpaydin, Section 8.2
"Support vector machines", from IEEE Intelligent Systems, July/August 1988.
Optional Readings
Tutorial Notes on "Support Vector and Kernel Machines" by Nello Cristianini, ICML 2001, Williams College
Course notes by Andrew Ng at Stanford
"A Tutorial on Support Vector Machines for Pattern Recognition" by Christopher Burges, Data Mining and Knowledge Discovery, June 1998
"Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines", by John Platt, April 1998.

Week of 10/31 Evaluation Methodology

Assignment writeup
Readings
Alpaydin: Chapter 14
Mitchell: Chapter 5
"The Case Against Accuracy Estimation for Comparing Induction Algorithms", ICML-98.

Week of 11/7 Learning Theory

Assignment writeup
Readings
Mitchell: Chapter 7 up to page 220
Kearns and Vazirani: pages 22-24
"Why Skewing Works: Learning Difficult Boolean Functions with Greedy Tree Learners", ICML-05.

Week of 11/14 Bias/Variance Theory and Ensemble Methods

Assignment writeup
Readings
"A Short Introduction to Boosting" by Freund and Schapire, 1999
Alpaydin: Sections 15.1, 15.2, 15.4, 15.5; also Sections 4.3 and 4.7
Duda, Hart, and Stork: Sections 9.5.1 and 9.5.2
"A decision-theoretic generalization of on-line learning and an application to boosting" by Freund and Schapire, 1995
"An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization" by Dietterich, Machine Learning Journal, 2000.

Week of 11/21 No meeting this week - Thanksgiving
Week of 11/28 Clustering

Assignment writeup
Readings
"Text Classification from Labeled and Unlabeled Documents Using EM" by Nigam, McCallum, Thrun, and Mitchell, Machine Learning, 2000
Mitchell, Section 6.12
Alpaydin, pages 140-144
Witten and Frank, pages 137-138, 262-266, 337-338
Bishop, 187-190, 65-72.

Week of 12/5 Presentations: Favorite Applications

Assignment writeup