And Behind Door No. 1, a Fatal Flaw

[Chuck Nessa]

I think there is a brand new DeSoto behind door number 2.

[JSngry]

And tell them Groucho told you to pick that curtain!

[John L]

I think that the disconnect here may be that Jim is not arguing about the statistical logic that we keep defending, but that a subjective hunch about something might have real meaning. What proof is there that a hunch is not based on some sort of ESP that we don't understand? Some people who feel lucky in blackjack might take a hit on 18 in hopes of getting and ace, a two, or a three. That is always stupid, given the odds. But if gamblers played the odds, they wouldn't be playing blackjack in the first place.

Jim's last comment about observing the past in order to assess the liklihood of what might happen in the future reminds me of a friend of mine who plays kino in casinos. First, he spends about 6 hours studying what numbers are coming up, tries to draw patterns, and then plays. No matter how many times I tell him that every kino game is completely independent of the past, he tells me to get fucked. But who really knows? Maybe mother nature, or some other supernatural force, really is affecting the history of the game in some systematic way that we don't understand? Or maybe it is even simpler than that. Some of the ping pong balls might not be of the exact same weight, and therefore come up with a different probability than the others.

The Let's Make a Deal example is interesting in that it is not the case that we know mother nature will choose one of three doors, each with 1/3 probability, and put a car there. There car is already there, and it was put there intentionally by Monty. So a gambler will try to conjure up some power to feel where it is. What is important to realize is that the assumption that this event is completely independent of past history might not be true. It is Monty, not mother nature, who decides where to put the car, and he is probably deliberately mixing it up so that one door does not get the car several times in a row. If this is the case, the fact that it was behind door number 2 last time should mean that it is less likely to be there this time. If it was behind door number 2 two or three times in a row already, then Monty would probably choose that door again with very low probability. The initial odds are then different than 1/3, 1/3, 1/3

In an earlier post, I argued that, even in the case that you have a strong prior hunch about which door the car is behind, you should still switch. The oprtimal strategy is be to make an initial choice of the door that you expect to be least likely, and then switch.

Jim responded that I wasn't accounting for goat farts, and he was right. The difference between a goat fart and a prior hunch is that the goat might fart only after the initial choice of a door has been made. So suppose you begin with an initial assessment of 1/3, 1/3, 1/3 and choose door number 1. Then Monty shows you a goat behind door number 2 and asks you if you want to switch to door number 3. Then you smell a strong goat fart. It might have come from that goat that Monty showed you, but you have a hunch that it is coming from door number 3 with high probability. On the basis of this fart, you update your subjective odds to 60/40 that it is behind door number 1 and you don't switch.

Is that a faithful representation of what you are trying to say, Jim?

Edited April 13, 2008 by John L

[Van Basten II]

I know one thing about these odds If you change your pick after Monty showed you the door with the donkey and you end up on the losing side. Ther's a 100 % chance that you'll get call a dumbass when you go back home.

[JSngry]

I think that the disconnect here may be that Jim is not arguing about the statistical logic that we keep defending, but that a subjective hunch about something might have real meaning. What proof is there that a hunch is not based on some sort of ESP that we don't understand? Some people who feel lucky in blackjack might take a hit on 18 in hopes of getting and ace, a two, or a three. That is always stupid, given the odds. But if gamblers played the odds, they wouldn't be playing blackjack in the first place.

Jim's last comment about observing the past in order to assess the liklihood of what might happen in the future reminds me of a friend of mine who plays kino in casinos. First, he spends about 6 hours studying what numbers are coming up, tries to draw patterns, and then plays. No matter how many times I tell him that every kino game is completely independent of the past, he tells me to get fucked. But who really knows? Maybe mother nature, or some other supernatural force, really is affecting the history of the game in some systematic way that we don't understand? Or maybe it is even simpler than that. Some of the ping pong balls might not be of the exact same weight, and therefore come up with a different probability than the others.

The Let's Make a Deal example is interesting in that it is not the case that we know mother nature will choose one of three doors, each with 1/3 probability, and put a car there. There car is already there, and it was put there intentionally by Monty. So a gambler will try to conjure up some power to feel where it is. What is important to realize is that the assumption that this event is completely independent of past history might not be true. It is Monty, not mother nature, who decides where to put the car, and he is probably deliberately mixing it up so that one door does not get the car several times in a row. If this is the case, the fact that it was behind door number 2 last time should mean that it is less likely to be there this time. If it was behind door number 2 two or three times in a row already, then Monty would probably choose that door again with very low probability. The initial odds are then different than 1/3, 1/3, 1/3

In an earlier post, I argued that, even in the case that you have a strong prior hunch about which door the car is behind, you should still switch. The oprtimal strategy is be to make an initial choice of the door that you expect to be least likely, and then switch.

Jim responded that I wasn't accounting for goat farts, and he was right. The difference between a goat fart and a prior hunch is that the goat might fart only after the initial choice of a door has been made. So suppose you begin with an initial assessment of 1/3, 1/3, 1/3 and choose door number 1. Then Monty shows you a goat behind door number 2 and asks you if you want to switch to door number 3. Then you smell a strong goat fart. It might have come from that goat that Monty showed you, but you have a hunch that it is coming from door number 3 with high probability. On the basis of this fart, you update your subjective odds to 60/40 that it is behind door number 1 and you don't switch.

Is that a faithful representation of what you are trying to say, Jim?

Closer than most, I guess...

I don't seem to be communicating my ideas successfully, so I'll stop (hopefully) after this. Suffice it to say that this:

think that the disconnect here may be that Jim is not arguing about the statistical logic that we keep defending

is exactly right, and that this

a subjective hunch about something might have real meaning.

might be better expressed as I have a hunch that there is a macro-pattern to how/where/when the 2/3 chance will yield a win & how/where/when the 1/3 chance will yield a win that is apparently as of yet outside the realm of "accepted" and/or existing statistical analysis, or even consideration therein.

Now yes, I know - every game has the same equal chance for the 2/3 chance to yield a win & for the 1/3 chance will yield a win, and in that proportion, so please, no repeats on that one, ok? But I'm not talking about chances, I'm talking about results, and the fact that, not just in this game but in all games of chance, some people consistently outperform/underperform against the odds, even when systematically making "counter-logical" and/or "smart choices" suggests to me that there is some sort of macro logic or macro pattern playing itself out.

You can call that "superstition" or "ignorance" or "naivite" or whatever, but I humbly (sic) suggest that outside of every observable level of randomness there lies an even greater order, and that outside of it lies a yet even greater level of randomness, etc etc etc ad infinitum. It is to that speculation/hunch/whatever that I have been addressing these comments. Apparently there is no respect for this notion, and ok. Hey, y'all got training and shit on this stuff that I don't and undoubtedly y'all know what you know, and I fully respect that.

All I do know is that I never fail to be surprised when science discovers new levels of order, nor am I surprised when science later finds new levels of chaos outside/inside the same. That cycle goes on forever, and it is one of the great joys to be had in this life, if you go that way, which I do.

"Playing hunches" may or may not be a fool's game. I've seen more people get burned more times than not, but there are those who continuously get it right. But sitll, it ain't something I'd ahng my hat on, not unless it was one that I had no pangs about losing. But looking for/considering the possibility of a macro order or logic as it pertains to "the odds playing out over time" doesn't seem at all far-fetched to me, even though I don't have anywhere near the "tools" to go about discovering/proving/whatever it.

I will also posit that at least as much as "the odds" are here to serve us, that we are also here to serve the odds, which means...whatever you want it to mean, I guess. But I do believe that this shit is far from unilateral.

Edited April 13, 2008 by JSngry

[The Magnificent Goldberg]

I was thinking something similar to what John wrote last night.

This game, as played on a computer, is properly random. As played live on TV, there are two (at least) embuggerance factors. One, which John mentioned, is that Monty may be deciding, so the choice of which door conceals the car is not totally random, and can therefore be analysed. The other is that, in any event, Monty knows which is the correct door. Someone who really pays attention to body language and voice tones may find clues there to change those odds; that's what professional poker players do, I understand, because there's really no such thing as a (perfect) poker face. So, as with the monkeys and the M&Ms, it is not true that, in the real world, everything about this issue is perfectly known and above-board.

Of course, I've never seen the show. It may be that it's set up in such a way that Monty only APPEARS to know which door conceals the car. So, perhaps he says something like, "and we'll see what's behind THIS door", presses the button - the only button he has - and the correct door opens because it's actually controlled by some other person. If it's set up that way, then there seems to return the notion to pure randomness.

MG

[JSngry]

In terms of pure randomness, what's the difference between Monty deciding where the car goes and some other human?

[John L]

Now yes, I know - every game has the same equal chance for the 2/3 chance to yield a win & for the 1/3 chance will yield a win, and in that proportion, so please, no repeats on that one, ok? But I'm not talking about chances, I'm talking about results, and the fact that, not just in this game but in all games of chance, some people consistently outperform/underperform against the odds, even when systematically making "counter-logical" and/or "smart choices" suggests to me that there is some sort of macro logic or macro pattern playing itself out.

You can call that "superstition" or "ignorance" or "naivite" or whatever, but I humbly (sic) suggest that outside of every observable level of randomness there lies an even greater order, and that outside of it lies a yet even greater level of randomness, etc etc etc ad infinitum. It is to that speculation/hunch/whatever that I have been addressing these comments. Apparently there is no respect for this notion, and ok. Hey, y'all got training and shit on this stuff that I don't and undoubtedly y'all know what you know, and I fully respect that.

All I do know is that I never fail to be surprised when science discovers new levels of order, nor am I surprised when science later finds new levels of chaos outside/inside the same. That cycle goes on forever, and it is one of the great joys to be had in this life, if you go that way, which I do.

"Playing hunches" may or may not be a fool's game. I've seen more people get burned more times than not, but there are those who continuously get it right. But sitll, it ain't something I'd ahng my hat on, not unless it was one that I had no pangs about losing. But looking for/considering the possibility of a macro order or logic as it pertains to "the odds playing out over time" doesn't seem at all far-fetched to me, even though I don't have anywhere near the "tools" to go about discovering/proving/whatever it.

I will also posit that at least as much as "the odds" are here to serve us, that we are also here to serve the odds, which means...whatever you want it to mean, I guess. But I do believe that this shit is far from unilateral.

Well, I think that we will have to agree to disagree about this. Genuine randomness is indeed chaotic, and the odds don't guarantee anything. We might have hunches that we want to go on that we don't completely understand. Fine. We can represent all of that by our own subjective probabilities about the odds, and play those odds in order to maximize our chances. "Odds about the odds" can be factored into all of this too. That is just semantics. Rational decision-making is an advantage. With chaotic randomness, it is no guarantee, and low probability events are realized all the time. It is an advantage. That's all.

[The Magnificent Goldberg]

In terms of pure randomness, what's the difference between Monty deciding where the car goes and some other human?

None - but if it's decided by pulling a number off a random number generator or table, then it's random (or as near as those tables can get to it).

MG

[JSngry]

Rational decision-making is an advantage. With chaotic randomness, it is no guarantee, and low probability events are realized all the time. It is an advantage.

Only when it works.

Seriously, though, I can no longer believe in total randomness, especially after having spent a few years trying to pursue it in music. Order of some sort inevitable asserts itself, although sometimes it takes a near-inhuman amount of detachment to recognize it. And where there's one level of order...there's another of chaos. and where there's one level of chaos, there's another of order. It's a spiral. Or something. And does it ever end/resolve/whatever? I don't think so.

Wasn't it Cecil Taylor who said that there's no such thing as "randomness" in music, especially if the music comes from the gut? Or something like that. I have to agree.

Now as it pertains to everyday stuff stuff, sure. I'll take all this and use it as it's meant to be used. But when "pondering", well... I wonder...

[Tom Storer]

Thanks for the summation of what we all already know! :tup :tup

Yeah, well, Jim, if you already knew it, you wouldn't ask "do you only look at the action going on in this room to figure that out, or do you look at everybody who's played the game in the past?" or say "2/3 vs 1/3 ain't necessarily gonna mean squat when your number comes up." That you make those remarks belies your assertion that you do understand the probability!

My take on goat farts is that a goat fart is additional information and changes the odds, just as Monty opening the door on one of the goats changes the odds. Same for hunches--they may be based on conscious or unconscious assimilation of further information, in which case they don't prove anything as concerns further levels of chaos and order as yet unsuspected by man. Hunches not based on assimilation of further information are called "guesses" no matter how strongly we believe in them.

[aparxa]

I might be too down-to-earth to conceive these philisophical issues. For me, this thread diverted from a mistake to questionable considerations. In practice, as humans or even machines can not simulate perfectly random numbers, we will probably never observe the 2/3 proportion. In theory, if a mathematical modelling leads to the 2/3 proportion, I don't see any reason to assume differently.

[Dan Gould]

"Playing hunches" may or may not be a fool's game. I've seen more people get burned more times than not, but there are those who continuously get it right.

I think that this is the crux of the matter for you, Jim, and I have to ask - do you really know people who "continuously get it right"? I believe there is a psychological term for the tendency to remember all the times that a hunch played out and forget all of the times that it didn't.

The fact is that your lucky friends and your unlucky friends are going to have the same odds - 1/3 if they stick, 2/3 if they switch.

[Dan Gould]

In practice, as humans or even machines can not simulate perfectly random numbers, we will probably never observe the 2/3 proportion.

Because I've too much time on my hands I wrote a quick and dirty Monty Hall simulation (Linux and Mac users need Mono, Windows users need to have .NET installed to run it) for those who only believe what they can see . See attachment.

Here's the output for 10000000 iterations:

10000000 iterations

Wins without changing doors: 3334363, 1/2.99907358616923

Wins with changing doors: 6665637, 1/1.50023171078773

On a percent basis, RC's simulation has a win when staying with your original door:

0.3334363

Changing doors:

0.6665637

I'd say that's pretty damn close to 1/3 and 2/3.

[aparxa]

I added the word perfectly

[John L]

"Playing hunches" may or may not be a fool's game. I've seen more people get burned more times than not, but there are those who continuously get it right.

I think that this is the crux of the matter for you, Jim, and I have to ask - do you really know people who "continuously get it right"? I believe there is a psychological term for the tendency to remember all the times that a hunch played out and forget all of the times that it didn't.

The fact is that your lucky friends and your unlucky friends are going to have the same odds - 1/3 if they stick, 2/3 if they switch.

Yes. An interesting aspect of genuine independent randomness is that it does not usually mix things up in the sense that the intuition of some people expect. If someone wins and wins again, then he or she just might win again a third time, even if the odds are against it. Genuine randomness is completely consistent with generating winning streaks or losing streaks that are very low probability events, and lead to the possible impression that something else is going on. In fact, genuine randomness will always create streaks from time to time. If we introduce something to the stochastic process to force the randomness to mix it up more and make streaks less plausible, then we are no longer dealing with genuine independent randomness.

[aparxa]

Random Number generators are really really effective, but there is still no generator where we can prove there is no kind of pattern in it.

I might be wrong.

[JSngry]

"Playing hunches" may or may not be a fool's game. I've seen more people get burned more times than not, but there are those who continuously get it right.

I think that this is the crux of the matter for you, Jim, and I have to ask - do you really know people who "continuously get it right"? I believe there is a psychological term for the tendency to remember all the times that a hunch played out and forget all of the times that it didn't.

Continuously, well, only my mother-in-law. She's spooky that way (and others...). LTB & I have been pleading with her for years to go to Vegas, but religious considerations won't let her even contemplate the notion. Sit down with her for a "fun" game at home, though, and you can't beat her, no matter what. She has rabbits in hats like no other...

But I also know a few people who, while not "perfect", routinely defy the odds to their benefit. But only a very few. And that goes towards "hunches", which I wonder (note - wonder only) are indicators of a greater, as of yet not understood, logic. Mr. Litwack says no to the logic, but maybe to the validity of the hunches, and for that I thank him for at least being open to the posibility of something other than a rigid "predestination" of sorts.

Similarly, I know a few, only a few, people who seem to never get right. They do everything perfectly and yet fail.

Them, I never loan money....

The fact is that your lucky friends and your unlucky friends are going to have the same odds - 1/3 if they stick, 2/3 if they switch.

Yes, Dan, I know.

[JSngry]

"Playing hunches" may or may not be a fool's game. I've seen more people get burned more times than not, but there are those who continuously get it right.

I think that this is the crux of the matter for you, Jim, and I have to ask - do you really know people who "continuously get it right"? I believe there is a psychological term for the tendency to remember all the times that a hunch played out and forget all of the times that it didn't.

The fact is that your lucky friends and your unlucky friends are going to have the same odds - 1/3 if they stick, 2/3 if they switch.

Yes. An interesting aspect of genuine independent randomness is that it does not usually mix things up in the sense that the intuition of some people expect. If someone wins and wins again, then he or she just might win again a third time, even if the odds are against it. Genuine randomness is completely consistent with generating winning streaks or losing streaks that are very low probability events, and lead to the possible impression that something else is going on. In fact, genuine randomness will always create streaks from time to time. If we introduce something to the stochastic process to force the randomness to mix it up more and make streaks less plausible, then we are no longer dealing with genuine independent randomness.

Now this, I like.

[Dan Gould]

"Playing hunches" may or may not be a fool's game. I've seen more people get burned more times than not, but there are those who continuously get it right.

I think that this is the crux of the matter for you, Jim, and I have to ask - do you really know people who "continuously get it right"? I believe there is a psychological term for the tendency to remember all the times that a hunch played out and forget all of the times that it didn't.

Continuously, well, only my mother-in-law. She's spooky that way (and others...). LTB & I have been pleading with her for years to go to Vegas, but religious considerations won't let her even contemplate the notion. Sit down with her for a "fun" game at home, though, and you can't beat her, no matter what. She has rabbits in hats like no other...

But I also know a few people who, while not "perfect", routinely defy the odds to their benefit. But only a very few. And that goes towards "hunches", which I wonder (note - wonder only) are indicators of a greater, as of yet not understood, logic. Mr. Litwack says no to the logic, but maybe to the validity of the hunches, and for that I thank him for at least being open to the posibility of something other than a rigid "predestination" of sorts.

Similarly, I know a few, only a few, people who

seem to never get right. They do everything perfectly and yet fail.

Them, I never loan money....

The fact is that your lucky friends and your unlucky friends are going to have the same odds - 1/3 if they stick, 2/3 if they switch.

Yes, Dan, I know.

Yes, you know that, but from the foregoing, would you say that it applies to your Mother-in-Law? I mean, you ascribe to her remarkable talents, "spooky" talents. If you show her the NYT simulator, so that no car is truly at stake, can she beat it in any real way? As in, play a hundred times with the "stay" strategy - does she break 40% or 50%? I think after 100 plays the tendency toward 1/3 - 2/3 will almost always be apparent. So if she could beat it by any significant margin, then I'd start to consider that you may have something.

Edited April 13, 2008 by Dan Gould

[JSngry]

Yes, you know that, but from the foregoing, would you say that it applies to your Mother-in-Law? I mean, you ascribe to her remarkable talents, "spooky" talents. If you show her the NYT simulator, so that no car is truly at stake, can she beat it in any real way? As in, play a hundred times with the "stay" strategy - does she break 40% or 50%? I think after 100 plays the tendency toward 1/3 - 2/3 will almost always be apparent. So if she could beat it by any significant margin, then I'd start to consider that you may have something.

Well, if she played the same strategy every time, yeah, she'd break 2/3-1/3. but if she just did her thing and went with her hunches in spite of the "smart choice" strategy, I bet she'd win more than 2/3-1/3, which again, is a matter of successfully knowing/feeling/guessing when to stay and when to switch, which has nothing to do with the always equal odds for each choice on every game.

[rockefeller center]

Yes, you know that, but from the foregoing, would you say that it applies to your Mother-in-Law? I mean, you ascribe to her remarkable talents, "spooky" talents. If you show her the NYT simulator, so that no car is truly at stake, can she beat it in any real way? As in, play a hundred times with the "stay" strategy - does she break 40% or 50%? I think after 100 plays the tendency toward 1/3 - 2/3 will almost always be apparent. So if she could beat it by any significant margin, then I'd start to consider that you may have something.

Well, if she played the same strategy every time, yeah, she'd break 2/3-1/3. but if she just did her thing and went with her hunches in spite of the "smart choice" strategy, I bet she'd win more than 2/3-1/3, which again, is a matter of successfully knowing/feeling/guessing when to stay and when to switch, which has nothing to do with the always equal odds for each choice on every game.

I guess it's time for James Randi to join this board. Holy crap.

[Tom Storer]

Well, if she played the same strategy every time, yeah, she'd break 2/3-1/3. but if she just did her thing and went with her hunches in spite of the "smart choice" strategy, I bet she'd win more than 2/3-1/3, which again, is a matter of successfully knowing/feeling/guessing when to stay and when to switch, which has nothing to do with the always equal odds for each choice on every game.

In a case like this Monty Hall game, what, besides the odds, could give you an advantage in choosing when to stay and when to switch? There isn't a way to successfully know, feel or guess when to stay and when to switch, unless you find further information such as a goat fart. If your mother-in-law is perceptive enough to pick up on clues, such as goat farts, Monty's eye movements, tire tracks in the dust leading to a particular door, etc., then yes, she can have better odds. Otherwise, you're talking about the supernatural, in which case why care about the odds at all?

[JSngry]

I don't believe in the supernatural. Simple as that.

I do, however, allow for the possibility that there are levels of "natural" that we have yet to discover. History/evolution would support this, no?

Or do you think that we've discovered/uncovered all there is to know?

Now as far as my mother-in-law goes, I believe that she is perceptive enough to pick up on those "other clues", but quite possibly/most likely subconsciously. That's why she's hard to beat at damn near any game involving anything even remotely involving "bluffing" or "calling dares". That's in no way supernatural, but it is a quality I've noticed that some people possess in greater quantity that others.

Now, my question from the beginning has been - is there a way to better develop those "gifts" of perception?

[Tom Storer]

I do, however, allow for the possibility that there are levels of "natural" that we have yet to discover. History/evolution would support this, no?

Or do you think that we've discovered/uncovered all there is to know?

Not at all, but I don't think that the fact that some people's hunches seem to come off more than some other people's hunches is a reason to think that existing logical explanations don't cover it.

Now, my question from the beginning has been - is there a way to better develop those "gifts" of perception?

So that's what your question has been from the beginning.