6.863J/9.611J Natural Language Processing: Fall 2012
 
 
Course home
[  Main  ] [  About ] [ Assignments ]
 

Staff
Prof. Robert C. Berwick
berwick@csail.mit.edu
32-D728, x3-8918
Office hours: Weds. 1-3pm

Course Support
Lynne Dell
lynne@mit.edu
32-D724, 617-324-1543
TA: Geza Kovacs
gkovacs@mit.edu
32-TBA
Office hrs: TBA

Course Time & Place
Lectures: M, W 11-12:30 PM
Room: 32-145,  map

Level & Prerequisites
Undergrad/Graduate; 6.034 or permission of instructor

Policies
Textbooks & readings
Grading marks guide
Style guide

Course Description

A laboratory-oriented course in the theory and practice of building computer systems for human language processing, with an emphasis on how human knowledge of language can be integrated into natural language processing.

This subject qualifies as an Artificial Intelligence and Applications concentration subject, Grad H level credit.
This term it also qualifies as a course 6 AUS subject.

Textbooks required for puchase or reference:
Jurafsky, D. and Martin, J.H., (JM) Speech and Language Processing (on library reserve, Barker P98.J87 2009)
2nd edition, Prentice-Hall: 2008. 
Some of the chapters from the revised edition may be posted in pdf form, as per the schedule shown on the homepage.
Additional textbook readings in: Manning & Schütze, (MS) Statistical Natural Language Processing, available online from MIT Cognet here.

Announcements:
• Week 1: Reading & response 1, available here, draft due Sunday 6pm & final version after Monday in-class discussion.

• Week 1: Fun NLP link of the week: Postmodernist paper generator. Try 'writing' a new paper by following this link.
• Week 1: And then, if you think the 'hard' sciences are immune, you can follow this link


Class days in blue, holidays in green, reg add/drop dates in orange.

September 2012
Sun
Mon
Tue
Wed
Thu
Fri
Sat
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23 24 25 26 27 28 29
30
October 2012
Sun
Mon
Tue
Wed
Thu
Fri
Sat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31  
           
November 2012
Sun
Mon
Tue
Wed
Thu
Fri
Sat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28 29 30  
             
December 2012
Sun
Mon
Tue
Wed
Thu
Fri
Sat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19 20 21 22
23
24 25 26 27 28 29
30            
Course schedule at a glance
Date
Topic
Slides & Reference Readings
Laboratory/Assignments

9/5
Weds

Introduction: walking the walk, talking the talk
Lecture 1 pdf slides; pdf 4-up; Jurafsky & Martin (JM), ch. 1.
If you don't know Python, read the NLTK book, ch. 1-3; otherwise, skim NLTK book, chs 2–3.
Background Reading (for RR 1): Jurafsky & Martin ch.4 on ngrams. (pp. 83-94; p. 114-116)
Background Reading (for RR 1): Abney on statistics and language.
Background Reading (for RR 1): Chomsky, Extract on grammaticality, 1955.
(Optional) Idealizations in science, by Cartwright
Background chapters on NLP from Russell & Norvig, ch. 22.
Assignment 1, Reading & response (RR 1) OUT
(Ngrams; NLTK Python warmup)

9/10
Mon
RR1 discussion Language models & ngrams
• Lecture 2 pdf slides; pdf 4-up
• Manning & Shütze (MS), ch. 2 (Review of probability)

Reading & response 1
DRAFT DUE SUN 6 PM; IN-CLASS DISCUSSION & FINAL TURN-IN MON
Assignment 2, Context-free grammars, OUT

9/12
Weds
Language models: Smoothing & Part of speech tagging

• Lecture 3 pdf slides; pdf 4-up
• MS, ch. 10

 
9/17
Mon

Context-free parsing I

Lecture 4 pdf slides; pdf 4-up;
JM, ch. 13 (parsing), pp. 427-435; ch. 14, pp. 459-467

• Handout on context-free grammars & parsing




9/19
Weds
Context-free parsing II: intro to statistical CF parsing
• Lecture 5 pdf slides; pdf 4-up



9/24
Mon

Competitive grammar writing (CGW) exercise

• Lecture 6 pdf slides: Competitive Grammar Writing slides, pdf; pdf, 2uppdf 4-up

Assignment 2, Context-free grammars, DUE
Assignment 3: Competitive Grammar Writing (CGW): teams assigned; CGW checkpoint OUT MON
Read CGW handout
CGW Checkpoint DUE FRI


9/26
Weds
Competitive grammar writing I
Bring notebook computer to class (at least 1 per team)

Competitive Grammar Writing

10/1
Mon
Competitive grammar writing II
Bring notebook computer to class (at least 1 per team)


Competitive Grammar Writing: Grammars FROZEN
10/3
Weds
Competitive Grammar Evaluation & Wrap-up, Grammy Awards
Awards & Discussion of results; can you beat a computer?


Competitive Grammar AWARDS
10/8
Mon
Language models, HMM tagging
Lecture 7 pdf slides; pdf 4-up

Assignment 4, Language Models, OUT
10/10
Weds
Language Models, log-linear modes
• Lecture 8 pdf slides; pdf 4-up


10/17
Weds
How do children recognize words?

Lecture 9 pdf slides; pdf 4-up

Assignment 5, RR #2 OUT
Reading: Saffran & Newport
10/22
Mon

RR 2 discussion; word parsing I

• Lecture 10 pdf slides; pdf 4-up; animation of Earley algorithm here; 4upbw Earley here.

Assignment 4, Language Models, DUE

10/24
Weds
Modern statistical parsers I
• Lecture 11 pdf slides; pdf 4-up
• JM ch. 14
• Background Reading: deMarcken, Lexical heads, phrase structure, & the induction of gramma 1995.
• Background Reading: Collins, Head-driven statistical models for natural language processing, 2003.


10/29
Mon
Treebank parsers II
• Lecture 12 pdf slides; pdf 4-up


10/31
Weds
Semantics I: the lambda calculus view
Lecture 13 pdf slides; pdf 4-up

Assignment 6, Word parsing, DUE
Assignment 7, Statistical Parsers, OUT

11/5
Mon
Semantics II: SQL
Lecture 14 pdf slides; pdf 4-up

Project proposal checkpoint: paragraph on team & proposal

11/7
Weds

 

Lecture 15 pdf slides; pdf 4-up



11/12
Mon
Lecture 16 pdf slides; pdf 4-up



11/14
Weds
Lecture 17 pdf slides; pdf 4-up

Assignment 7, Statistical Parsers, OUT
Assignment
8, Lexical semantics, OUT

11/19
Mon
Lexical Semantics

Lecture 18 pdf slides; pdf 4-up
• Gleitman, Verb learning and the footprint of universal grammar, 2003.
• Brent, Automatic acquisition of verb subcategorization frames from untagged text

 
11/21
Weds
Discourse

• Lecture 17 pdf slides; pdf 4-up


11/26
Mon
Language Learning
• Lecture 18 pdf slides; pdf 4-up
• Background Reading: Niyogi & Berwick, A language learning model for finite parameter spaces, 1996.
Assignment 8, Lexical Semantics DUE
11/28
Weds
Language Learning & Language Change
• Lecture 19 pdf slides; pdf 4-up
Background Reading: Niyogi & Berwick, A dynamical systems model for language change, 1997.
12/3
Mon
Evolution of language
• Lecture 20 pdf slides; pdf 4-up
• Background Reading: Chomsky, Fitch, Hauser, The Faculty of Language
Background Reading: Berwick, Syntax Facit Saltum, 2008.
 
12/5
Weds
Evolution of language
• Lecture 21 pdf slides; pdf 4-up
12/10
Mon
  • Lecture 22 pdf slides; pdf 4-up  
12/12
Weds
  • Lecture 23 pdf slides; pdf 4-up Final Projects DUE
 

 

MIT Home