´ºË®ÌÃÊÓƵ

Spring 2018 Instructor: Michael Mozer

Course Objectives

For humans and machines, intelligence requires making sense of the world — inferring simple explanations for the mishmosh of information coming in through our senses, discovering regularities and patterns, and being able to predict future states. In artificial intelligence and cognitive science, the formal language of probabilistic reasoning and statistical inference have proven useful to model intelligence. From a probabilistic perspective, knowledge is represented as degrees of belief, observations provide evidence for updating one's beliefs, and learning allows the mind to tune itself to statistics of the environment in which it operates.

One virtue of probabilistic models is that they straddle the gap between cognitive science, artificial intelligence, and machine learning. The same methodology is useful for both understanding the brain and building intelligent computer systems. Indeed, for much of the research we'll discuss, the models contribute both to machine learning and to cognitive science. Whether your primary interest is in engineering applications of machine learning or in cognitive modeling, you'll see that there's a lot of interplay between the two fields.

The course participants are likely to be a diverse group of students, some with primarily an engineering/CS focus and others primarily interested in cognitive modeling (building computer simulation and mathematical models to explain human perception, thought, and learning).

Prerequisites

The course is open to any students who have some background in cognitive science or artificial intelligence and who have taken an introductory probability/statistics course or the graduate machine learning course (CSCI 5622). If your background in probability/statistics is weak, you'll have to do some catching up with the text.

Course Readings

We will be using the text Bayesian Reasoning And Machine Learning by David Barber (Cambridge University Press, 2012). The author has made available an electronic version of the text. Note that the electronic version is a 2015 revision. Because the electronic version is more recent, all reading assignments will refer to section numbers in the electronic version.

For additional references, wikipedia is often a useful resource. The pages on various probability distributions are great references. If you want additional reading, I recommend the following texts:

• Pattern Recognition and Machine Learning by Chris Bishop
• Bayesian Data Analysis by Gelman, Carlin, Stern, & Rubin
• Machine Learning: A Probabilistic Perspective by Kevin Murphy [be sure to get the fourth printing; there were many typos in earlier versions]
• Bayesian cognitive modeling: A practical course by Michael Lee and Erik-Jan Wagenmakers [electronic version online]
• We will also be reading research articles from the literature, which can be downloaded from the links on the class-by-class syllabus below.

Course Discussions

We will use Piazza for class discussion. Rather than emailing me, I encourage you to post your questions on Piazza. Feel free to post anonymously. I strive to respond quickly. If I do not, please email me personally. 

Course Requirements

Readings

In the style of graduate seminars, your will be responsible to read chapters from the text and research articles before class and be prepared to come into class to discuss the material (asking clarification questions, working through the math, relating papers to each other, critiquing the papers, presenting original ideas related to the paper).

Homework Assignments

We can all delude ourselves into believing we understand some math or algorithm by reading, but implementing and experimenting with the algorithm is both fun and valuable for obtaining a true understanding. Students will implement small-scale versions of as many of the models we discuss as possible. I will give about 10 homework assignments that involve implementation over the semester, details to be determined. Most students in the class will prefer to use python, and the tools we'll use are python based. If you have a strong preference, matlab is another option. For one or two assignments, I'll ask you to write a one-page commentary on a research article.

Semester Grades

Semester grades will be based 5% on class attendance and participation and 95% on the homework assignments. I will weight the assignments in proportion to their difficulty, in the range of 5% to 15% of the course grade. Students with backgrounds in the area and specific expertise may wish to do in-class presentations for extra credit.
Procedures for Homework Assignments

General Policy

You may work either individually or in a group of two. If you work with someone else, I expect a higher standard of work. I'm not proud to tell you this, but from 30 years of grading, I have to warn you that professors and TAs have a negative predisposition toward hand printed work. It is much easier to digest responses that are typed, spell corrected, and have made an effort to communicate clearly. We will be grading not only on the results you obtain but on the clarity of your write up.
Because of the large class size, no late assignments will be accepted without a medical excuse or personal emergency. If you have a conflicting due date in another class, give us a heads-up early and we'll see about shifting the due date.

Instructor and TA are eager to help folks who are stuck or require clarification. For any clarification of the assignment, what we're expecting, and how to implement, we would appreciate it if you post your question on piazza. In fact, post on piazza unless your question is personal or you believe it is specific to you. If you have the question, it's likely others will have the same question. And if we give you a clue, then we'll give the same clue to everyone else.
See additional information at the end of the syllabus on academic honesty.

Submission of Work

We ask you to submit a hardcopy of your write up (but not code) in class on the due date.
We also ask that you upload your write up and any code as a .zip file on moodle.
Be sure to write your full name on the hardcopy and in the code.
If you are working in a group, hand in only one hard copy and put both of your names on the write up and code.
We ordinarily will not look at your code, unless there appears to be a bug or other problem.