In response to the need for more rapid and iterative feedback on customer preferences, researchers are developing new web-based conjoint analysis methods that adapt the design of conjoint questions based on a respondent's answers to previous questions. Adapting within a respondent is a difficult dynamic optimization problem and, until recently, adaptive conjoint analysis' (ACA) utility-balance heuristic was the only widely-used method for addressing this adaptation. In this paper we apply and test a new polyhedral method that uses “interior-point” math programming techniques. This question-design method is benchmarked against both ACA and an efficient, non-adaptive design (Fixed).
Over 300 respondents were randomly assigned to different experimental conditions and were asked to complete a web-based conjoint exercise. The conditions varied based on the design of the conjoint exercise. Respondents in one group completed a conjoint exercise designed using the ACA method, respondents in another group completed an exercise designed using the Fixed method, and the remaining respondents completed an exercise designed using the polyhedral method. Following the conjoint exercise, respondents were given $100 and allowed to make a purchase from a Pareto choice set of five new-to-the-market laptop computer bags. The respondents received their chosen bag together with the difference in cash between the price of their chosen bag and the $100.
We compare the methods on both internal and external validity. Internal validity is evaluated by comparing how well the different conjoint methods predict several holdout conjoint questions. External validity is evaluated by comparing how well the conjoint methods predict the respondents' selections from the choice sets of five bags. Each method combines a modular question-design component with a modular estimation component, and so we conducted additional analysis to separately evaluate the accuracy of the question-design and estimation components. As part of this analysis we explored alternative estimation methods.
The results reveal a remarkable level of consistency across the two validation tasks. In the initial comparisons the polyhedral method was significantly more accurate than the Fixed method and offered similar performance to the ACA method. In our analysis of hybrid methods, the preferred estimation method varied based on the question-design method, although the Hierarchical Bayes estimation methods consistently performed well. The comparison of question-design methods offered strong evidence supporting polyhedral question-design over the Fixed and utility-balance (ACA) question-design methods. This finding holds across estimation methods.
Download Paper (pdf)
.
