What is the likelihood function? 

 An interesting 1992 paper by Bayarri and DeGroot entitled &#8220;Difficulties and Ambiguities in the Definition of a Likelihood Function&#8221; ( gated version ) grapples with the problem of defining the likelihood when auxiliary variables are at hand. Here is the abstract: 

 
   The likelihood function plays a very important role in the development of both the theory and practice of statistics. It is somewhat surprising to realize that no general rigorous definition of a likelihood function seem to ever have been given. Through a series of examples it is argued that no such definition is possible, illustrating the difficulties and ambiguities encountered specially in situations involving &#8220;random variables&#8221; and &#8220;parameters&#8221; which are not of primary interest. The fundamental role of such auxiliary quantities (unfairly called &#8220;nuisance&#8221;) is highlighted and a very simple function is argued to convey all the information provided by the observations. 
 

 The example that resonates with me in on pages 4-6, where they describe the ambiguity of using defining the likelihood function when there is an observation  y  which is a measurement of  x  subject to (classical) error. There are several different ways of writing a likelihood in that case, depending on how you handle the latent, unobserved data  x .  One can condition on it, marginalize across it, or include it in the joint distribution of the data. Each of these can lead to a different MLE.  

 Their point is that situations like this involve subjective choices (though, all modeling requires subjective choice) and the hermetic seal between the &#8220;model&#8221; and the &#8220;prior&#8221; is less airtight than we think.