Posts Tagged ‘InformationTheory’
[PyDataGlobal2025] Lessons in Decision Making from the Monty Hall Problem
Lecturer
Eyal Kazin is an ex-cosmologist turned data scientist with two decades of experience solving complex quantitative problems. Based in London, he applies statistical and causal inference, machine learning, and optimization methods to produce practical, communicable insights for organizational decision makers. He regularly publishes accessible articles on applied statistics and machine learning on Medium.
Abstract
The Monty Hall problem, popularized by a television game show and later by a magazine column, continues to serve as a rich source of insight into probabilistic reasoning. This article reconstructs the classic three-door formulation, contrasts frequentist and Bayesian analyses, quantifies the informational value of the host’s actions through the notion of surprisal, and extracts a series of decision-making lessons relevant to data science practice. Qualitative and quantitative criteria for recognizing genuine real-world analogues are examined, together with cautionary observations about over-simplification and the limits of analogy.
The Classic Problem and Competing Intuitions
A contestant faces three doors; a prize lies behind one and undesirable objects behind the others. After the contestant selects a door, the host—who knows the prize location—opens a different door that does not conceal the prize and offers the contestant the chance to switch to the remaining closed door. The question is whether switching improves the probability of winning.
A frequentist intuition notes that only two doors remain and therefore assigns equal probability to each, concluding that switching is irrelevant. A Bayesian analysis begins with a uniform prior of one-third on each door. After the contestant’s initial choice, the complementary probability mass of two-thirds resides on the other two doors collectively. When the host reveals one of those doors to be empty, the entire two-thirds mass concentrates on the single remaining unchosen door. Switching therefore doubles the probability of success from one-third to two-thirds.
The same logic becomes intuitively transparent when the number of doors is increased to one hundred or one million. The contestant’s initial selection retains probability (1/n); after the host opens all but one of the remaining doors, the complementary mass ((n-1)/n) resides on the single unopened alternative. The dramatic disparity renders the advantage of switching obvious. Information-theoretic surprisal formalizes the difference: the host’s action conveys progressively more information as (n) grows, widening the gap between the surprisal of the originally chosen door and that of the residual alternative.
Decision-Making Lessons for Data Practice
Several transferable lessons emerge. First, probability assessment is frequently counter-intuitive; Kahneman’s distinction between fast, intuitive System 1 thinking and slower, deliberative System 2 thinking maps directly onto the frequentist versus Bayesian treatments of the problem. Second, ambiguity in problem statements must be resolved explicitly; the host’s policy of never opening the prize door is rarely stated yet is essential to the correct solution. Third, new information should update beliefs rather than be discarded. Fourth, well-designed visualizations—such as exhaustive enumeration of the six equally likely game configurations—can convert an abstract argument into an immediately graspable demonstration. Fifth, multiple solution paths exist; Marilyn vos Savant’s public defense of the Bayesian answer, despite initial resistance from many professional mathematicians, illustrates both the value of alternative representations and the necessity of intellectual humility.
Simulations, while powerful for complex systems, are unnecessary when an exact analytic solution is available; the Monty Hall problem can be solved with pen and paper. Useful analogies should be sought when a problem appears intractable, yet the trade-off between simplification and oversimplification must be kept in view. Occam’s razor, Menken’s warning against excessive simplicity, and Box’s maxim that all models are wrong but some are useful together supply a balanced heuristic.
Real-World Analogues and the Limits of Analogy
Genuine applications must reproduce both the qualitative dependence structure and the quantitative conditional probabilities of the original problem. The host’s choice depends on both the contestant’s selection and the prize location; the two latter variables are independent of each other. Candidate analogies such as short-listing job applicants or adaptive tutoring succeed only when the “host” possesses complete knowledge of the target and the learner has essentially no prior information. Superficial similarities—three cups in a magic trick, for example—satisfy the formal conditions yet remain of limited practical interest. Causal graphs make the required independence and dependence relations explicit and thereby guard against misleading parallels.