# Modeling and Analysis in Experimental Semantics and Pragmatics Experimental approaches to theoretical questions in semantics and pragmatics are booming. Some see an empirical turn in progress. A welcome enrichment it may be, but the unification of rich theoretical work and novel experimental data brings new conceptual and practical problems: how do established theoretical notions lead to empirically testable predictions and what can we learn from experimental data about theoretical variables of interest? This course addresses these questions by introducing theory-driven probabilistic modeling in connection with Bayesian data analysis as a helpful set of tools to learn from observational data through the lens of a theoretical model. We will introduce the basics of Bayesian data analysis and probabilistic modeling through a series of concrete case studies in natural language semantics and pragmatics. - Instructors: Michael Franke and Michael Henry ("MH" or "Emaixe") Tessler - Time: 14:00 - 15:30 - Location: Room 272, Floor 3 Fill out the [end of course survey](https://docs.google.com/forms/d/e/1FAIpQLSdmczuCL1zEqhH2Sq1eBCMtTCLLSim0fV2Sy2jee8D-m5-1pg/viewform?usp=sf_link)! - Course Webbooks - [Bayesian Data Analysis using Probabilistic Programs](https://mhtess.github.io/bdappl/) - [Probabilistic Language Understanding](http://www.problang.org) ## Schedule Note that schedule is subject to change and materials are likely to be updated up until class time. - Day 1: Why analyze data: Inference, comparison, criticism [[slides]](http://www.mit.edu/~tessler/short-courses/2018-bdarsa-esslli/slides/03_stats.html) [[post-class question]](https://docs.google.com/forms/d/e/1FAIpQLScOuKsk-hDgmxLBkPhPoKXaQdME1ZMBp77JNSIONwDSCAWXeA/viewform?usp=sf_link) [[optional: bdappl i]](https://mhtess.github.io/bdappl/chapters/01-intro.html) - Day 2: Basics of probabilistic programming, or how to WebPPL [[bdappl ii]](https://mhtess.github.io/bdappl/chapters/02-introPPL.html) [[post-class question]](https://docs.google.com/forms/d/e/1FAIpQLSdNCU2z_09EUVb6uBHh-uTT-AEoNDow1BmoRKlAyHPIKJM1ng/viewform?usp=sf_link) - Day 3: Data analysis in WebPPL [[bdappl iii: parameter inference and model criticism]](https://mhtess.github.io/bdappl/chapters/03-simpleModels.html), [[bdappl iv: hypothesis testing]](https://mhtess.github.io/bdappl/chapters/04-hypothesisTesting.html) - Day 4: Analyzing a language model in WebPPL [[plu appendix iii]](http://www.problang.org/chapters/app-04-BDA.html) [[slides (monsters)]](http://stanford.edu/~mtessler/short-courses/2018-bdarsa-esslli/slides/03_monsters.html) - Day 5: Case study in how probabilistic programming enables creative analyses (time permitting) [[short paper on link functions (very last part of the session)]](http://www.home.uni-osnabrueck.de/michfranke/Papers/Franke_2016_Task%20types,%20link%20functions%20&%20probabilistic%20modeling%20in%20experimental.pdf) Answers to your questions are provided [here](http://stanford.edu/~mtessler/short-courses/2018-bdarsa-esslli/slides/02_QA.html). ## Useful resources #### WebPPL support and packages - [webppl.org](http://webppl.org): An online editor for WebPPL - [WebPPL documentation](http://webppl.readthedocs.io/en/master/) - [WebPPL dev Google Group](https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/webppl-dev): Public forum for discussing issues with WebPPL - [WebPPL-viz](http://probmods.github.io/webppl-viz/): A summary of the vizualization options in WebPPL - [RWebPPL](https://github.com/mhtess/rwebppl): If you would rather use WebPPL within R - WebPPL [packages](http://webppl.readthedocs.io/en/dev/packages.html) (e.g. csv, json, fs). - [A WebPPL package with useful BDA helper functions](https://github.com/mhtess/webppl-bda) #### Basic WebPPL tutorials - [WebPPL intro from DIPPL](http://dippl.org/chapters/02-webppl.html). - [WebPPL intro from AgentModels](http://agentmodels.org/chapters/2-webppl.html). #### Bayesian Data Analysis (using WebPPL) - [Probabilities and Bayes Rule in WebPPL](http://www.problang.org/chapters/app-01-probability.html) by Michael Franke - [Comparing methods for computing Bayes Factors](http://michael-franke.github.io/statistics,/modeling/2017/07/07/BF_computation.html) by Michael Franke - [A BDA course syllabus](http://web.stanford.edu/class/psych201s/) by MH Tessler #### Other WebPPL applications - [Probabilistic Models of Cognition](http://probmods.org/): An introduction to computational cognitive science and the probabilistic programming language WebPPL - [Design and Implementation of Probabilistic Programming Languages](http://dippl.org). - [Modeling Agents with Probabilistic Programs](http://agentmodels.org): An introduction to formal models of rational agents using WebPPL - [Forest](http://forestdb.org): A Repository for probabilistic models ### Great textbooks on Bayesian Data Analysis - [Doing Bayesian Data Analysis](https://sites.google.com/site/doingbayesiandataanalysis/) (Kruschke) - [Bayesian Data Analysis](http://www.stat.columbia.edu/~gelman/book/) (Gelman) - [Bayesian Cognitive Modeling](https://bayesmodels.com) (Lee & Wagenmakers)