The Myth of the Hot Hand - Mathematically Vindicated?

[Philadukie]

I don't pretend to fully understand the math behind it, but maybe some of our stats/math oriented posters would like to comment.

http://www.slate.com/articles/health...sis_shows.html

I've always thought hot hands exist, but I've learned to question over the years what I see with my eyes thanks to the stats folks on here!

[swood1000]

I don't pretend to fully understand the math behind it, but maybe some of our stats/math oriented posters would like to comment.

http://www.slate.com/articles/health...sis_shows.html

I've always thought hot hands exist, but I've learned to question over the years what I see with my eyes thanks to the stats folks on here!

I recently read Thinking, Fast and Slow, by Daniel Kahneman. It was an awesome book that I highly recommend. He also asserted that the "hot hand" is a cognitive illusion:

The "fact" that players occasionally acquire a hot hand is generally accepted by players, coaches, and fans. The inference is irresistible: a player sinks three or four baskets in a row and you cannot help forming the causal judgment that this player is now hot, with a temporarily increased propensity to score. Players on both teams adapt to this judgment — teammates are more likely to pass to the hot scorer and the defense is more likely to double-team. Analysis of thousands of sequences of shots led to a disappointing conclusion: there is no such thing as a hot hand in professional basketball, either in shooting from the field or scoring from the foul line. Of course, some players are more accurate than others, but the sequence of successes and missed shots satisfies all tests of randomness. The hot hand is entirely in the eye of the beholders, who are consistently too quick to perceive order and causality in randomness. The hot hand is a massive and widespread cognitive illusion.

This was the only thing in the book that I thought he was wrong on, simply because I myself have experienced being "hot" and "cold" in sporting events, although he would say that feeling "hot" is an illusion caused by hitting the mark more frequently in a given period of time, but this can be expected to occur randomly to everyone. Still not buying that it is an illusion.

[uh_no]

I don't pretend to fully understand the math behind it, but maybe some of our stats/math oriented posters would like to comment.

http://www.slate.com/articles/health...sis_shows.html

I've always thought hot hands exist, but I've learned to question over the years what I see with my eyes thanks to the stats folks on here!

Interesting.

For the interested, here's the paper:
http://poseidon01.ssrn.com/delivery...105027&EXT=pdf

will have to read later.

[Kedsy]

I don't pretend to fully understand the math behind it, but maybe some of our stats/math oriented posters would like to comment.

http://www.slate.com/articles/health...sis_shows.html

I've always thought hot hands exist, but I've learned to question over the years what I see with my eyes thanks to the stats folks on here!

I've always thought the original, seminal study was flawed on many levels. And I've always believed in the hot hand, even if it's spawned by what is essentially an illusion. If a player feels hot, he's more confident and less tight, and thus more likely to shoot with his optimal shooting motion. If a player feels cold, he'll hesitate or think too much or try too hard, with the result that he's less likely to use his optimal shooting motion. Thus, the feeling causes a self-fulfilling prophecy, and the player really is hotter or colder, depending on how he feels.

Whether or not a player makes his next shot has sooooo many variables that a single miss or make doesn't necessarily tell the whole story.

[MChambers]

I've always thought the original, seminal study was flawed on many levels. And I've always believed in the hot hand, even if it's spawned by what is essentially an illusion. If a player feels hot, he's more confident and less tight, and thus more likely to shoot with his optimal shooting motion. If a player feels cold, he'll hesitate or think too much or try too hard, with the result that he's less likely to use his optimal shooting motion. Thus, the feeling causes a self-fulfilling prophecy, and the player really is hotter or colder, depending on how he feels.

Whether or not a player makes his next shot has sooooo many variables that a single miss or make doesn't necessarily tell the whole story.

Kedsy,

I've noticed that if you have a good post, the next post is likely to be good as well. Haven't tracked it statistically, but I bet you tend to get sporked in tight clumps.

[swood1000]

I've always thought the original, seminal study was flawed on many levels. And I've always believed in the hot hand, even if it's spawned by what is essentially an illusion. If a player feels hot, he's more confident and less tight, and thus more likely to shoot with his optimal shooting motion. If a player feels cold, he'll hesitate or think too much or try too hard, with the result that he's less likely to use his optimal shooting motion. Thus, the feeling causes a self-fulfilling prophecy, and the player really is hotter or colder, depending on how he feels.

Right. It seems self-evident that this must have some bearing. I get the feeling that scientists who make these statements don't have any sports experience (although I don't think that this phenomenon is limited to sports).

[BD80]

I've always thought the original, seminal study was flawed on many levels. And I've always believed in the hot hand, even if it's spawned by what is essentially an illusion. If a player feels hot, he's more confident and less tight, and thus more likely to shoot with his optimal shooting motion. If a player feels cold, he'll hesitate or think too much or try too hard, with the result that he's less likely to use his optimal shooting motion. Thus, the feeling causes a self-fulfilling prophecy, and the player really is hotter or colder, depending on how he feels.

Whether or not a player makes his next shot has sooooo many variables that a single miss or make doesn't necessarily tell the whole story.

I agree with the hot hand concept. Anyone who has watched or played sports knows there is something to it.

Also agree that there are so many variables, including some raised by the very concept of "hot hand," such as tighter defense. IMO, the greatest counterbalance is shot selection. A "hot hand" often leads to "brain freeze" with respect to shot selection. The phenomenon gave birth to the phrase "heat check," in which the player takes more and more difficult shots, testing how "hot" he or she is.

[uh_no]

Right. It seems self-evident that this must have some bearing. I get the feeling that scientists who make these statements don't have any sports experience (although I don't think that this phenomenon is limited to sports).

A similar argument was made when sabermatricians when it was becoming popular...that they weren't "baseball" people...but it also doesn't mean they're wrong.

Statistical independence has a very precise mathematical definition (though in this case it seems they "did it wrong"). Calculating whether a series of events is independent doesn't depend on the domain. Further, not rejecting some null hypothesis also does not depend on the domain.

Where the domain knowledge becomes relevant from a statistical standpoint is the conclusions your draw from your calculations.

Anecdote:

In 2006 i had a fellow mets fan with whom i chatted on line who often watched mets games without pants on. The mets won the first 20 of their games when he wasn't wearing pants, and were something like 4-12 when he wore pants. Statistically, it was something like 99.7% likely that his pants explained the mets outcomes. That's the math. you can't argue with the math. But...his pants obviously have no effect on the mets...and you wouldn't know that unless you knew that a random guy wearing pants in brooklyn has no effect on a baseball game.


So...if made shots for an individual are demonstrated to be independent, they are independent (which this new study casts serious doubt on). Case closed...you can't argue with the definition and calculation (unless they did it wrong...). They don't need to know a darn thing about basketball to perform the calculation. I could pump the numbers into my computer and it could perform the calculation. It doesn't care if it's a coin toss, basketball shots, or the spin of a roulette wheel.

[gus]

I don't pretend to fully understand the math behind it, but maybe some of our stats/math oriented posters would like to comment.

http://www.slate.com/articles/health...sis_shows.html

I've always thought hot hands exist, but I've learned to question over the years what I see with my eyes thanks to the stats folks on here!

Interesting -- I'll have to read the paper later. From the slate article, I'd like to see the support for this statement:

the average shooting percentage following three misses will ... be higher than the average shooting percentage following three hits.

If it's just a point that averaging averages can be misleading, then I agree: but they're using that to justify a conclusion that the data presented in the Gilovich, Tversky and Vallone study therefore proves the hot hand... well, that seems like a problematic conclusion.

Regardless, even if a hot hand is mathematically proven, I think the conclusion from this article linked in the slate piece is probably correct:

My reasoning goes as follows. Gilovich et al. reported three things in their paper. First, that there’s no evidence for any hot hand in basketball shooting. I think they were wrong on this one; it does seem that, if you look at basketball data carefully, you do see evidence for a hot hand, it’s just that the original analyses were hampered by bias and variance issues. Second, Gilovich et al. report that basketball fans view the hot hand effect as huge, much larger than any such effect in reality. I find their results convincing on that point. It does seem that once a person focuses on any particular effect, once he or she believes it to be nonzero, there’s a tendency to overrate its importance. I guess that’s related to the “availability heuristic” studied by Amos Tversky, another author of that hot hand paper. Gilovich et al.’s third finding is that people perceive a hot hand even if you give them completely random sequences. That appears to be true too, even if not newsworthy on its own.

[dukelifer]

I don't pretend to fully understand the math behind it, but maybe some of our stats/math oriented posters would like to comment.

http://www.slate.com/articles/health...sis_shows.html

I've always thought hot hands exist, but I've learned to question over the years what I see with my eyes thanks to the stats folks on here!

All I know is the Laettner didn't know his stats. Clearly, ignorance is bliss for all of us Duke fans.

[cspan37421]

Let's be clear ... when a statistician says that there's no such thing as a hot hand in basketball, they are not referring to the feeling of confidence and fluidity of shooting motion that comes after a couple or few successful shots in a row. No statistician is denying that, or even commenting upon it. Nor are they saying that there aren't streaks of success or misses to be found ex post.

Instead, the question is more along the lines of, given a series of successful (unsuccessful) shots have just occurred, is the shooter more likely (less likely) to make the next shot than they normally would have been?

They may also be checking for serial correlation, etc.

So these are statistical questions. These have nothing to do with whether you've played the game or not, whether you've experienced that feeling that you can't miss, etc. So I think it's very important to understand what exactly is the claim being made, before dismissing it.

It's a bit like saying an active fund manager will not beat the market over time. Sure, in any given year, some will. But - unless their name is Buffett - that won't have any bearing on whether they beat it the following year. Take the top quartile of active managers in a given year. What are the odds they'll be top quartile again the following year? About 25%. What are the odds that a bottom quartile performer will be top quartile the following year? About 25%, at least, the last time I looked at it.

Beyond this test of randomness, suppose after 3 years you've got about 1/64th of active fund managers beating the market (assume these are corrected for leverage and risk, of course). Well, you might think, aha! The hot hand in investing! OK, fine. But just try to identify them in advance. There's the rub. So it's easy to spot the "hot hand" in retrospect. If you can spot it in advance, well, we may have to find you a coaching job.

[Kedsy]

So...if made shots for an individual are demonstrated to be independent, they are independent (which this new study casts serious doubt on). Case closed...you can't argue with the definition and calculation (unless they did it wrong...). They don't need to know a darn thing about basketball to perform the calculation. I could pump the numbers into my computer and it could perform the calculation. It doesn't care if it's a coin toss, basketball shots, or the spin of a roulette wheel.

Isn't this true only if you account for all the variables? Wouldn't basketball knowledge help you identify and account for them all? Although in this case, I would argue it is nearly impossible to account for everything.

[TruBlu]

Anecdote:

In 2006 i had a fellow mets fan with whom i chatted on line who often watched mets games without pants on. The mets won the first 20 of their games when he wasn't wearing pants, and were something like 4-12 when he wore pants. Statistically, it was something like 99.7% likely that his pants explained the mets outcomes. That's the math. you can't argue with the math. But...his pants obviously have no effect on the mets...and you wouldn't know that unless you knew that a random guy wearing pants in brooklyn has no effect on a baseball game.

You did not give adequate data. It depends on his location and other variables regarding him not wearing pants.

Was he at the Mets games sitting behind home plate while the opposing pitcher was on the mound, but moved to center field seats every time the Mets' pitcher was on the mound.
(This might be where Speedo Guy got his idea).

[-jk]

If I've stayed alive in a few rounds of craps, should I keep rolling? Or, if on the side, keep betting on him?

How 'bout that coin toss? How many times in a row is "hot"? What's the next flip?

We're pattern recognizing/seeking beasts. We find 'em everywhere we look.

(And, no, I don't believe in the "hot hand" regardless of how many FTs I've just hit in a row: maybe 2 or 3, occasionally 4 - but always with my pants on...)

-jk

[Kedsy]

If I've stayed alive in a few rounds of craps, should I keep rolling? Or, if on the side, keep betting on him?

How 'bout that coin toss? How many times in a row is "hot"? What's the next flip?

This would be relevant if the roller had control over how the dice landed. Shooting a basketball isn't like rolling dice. The shooter doesn't shoot exactly the same way every time. And if he does something differently while he's shooting, the result is affected. This isn't true for dice or coins. So the analogy doesn't really fly.

[uh_no]

Isn't this true only if you account for all the variables? Wouldn't basketball knowledge help you identify and account for them all? Although in this case, I would argue it is nearly impossible to account for everything.

I can take the string of 3 point shots for any player in his career and tell you whether they are independent or not. There is no variable to account for there...but as someone with domain knowledge, that number would be meaningless

It all comes down to the interpretation. Do you count shots across games? Do you count across halves? Do you account for how closely guarded the player is? I mean maybe that is accounting for variables.

But the point is the calculation itself is not accounting for anything. It all comes down to whether you think that calculation yields relevant data, and as you say, domain knowledge helps you identify that...and then only really bother putting weight in the calculations that DO yield relevant data.

You need both...you need stats, you need basketball people. It could be that all shots are independent, and that result could be meaningless. It bothers me a lot when people think there is this massive divide between stats and the "eye test". You need both...and you will be woefully incomplete without either.

As we regularly demonstrate here, often times people's impressions end up very wrong...but we also find that stats can't be trusted blindly...what stats do you even know are relevant without domain knowledge? As I pointed with the pants anecdote...without domain knowledge I would have thought no-pants == wins. I wouldn't say that was a variable that wasn't accounted for...but a stupid thing to look at in the first place! I can still say that at the 99.6% confidence level, pants have an effect...and I wouldn't be wrong.

[Mabdul Doobakus]

I know the "hot hand" has generally been dismissed by statisticians in the past, and I have in fact made arguments against it in the past. But, honestly, I don't see how it could not exist. If you've ever played golf, sometimes your swing is right, and sometimes it isn't. Same thing in basketball. Sometimes the ball comes off your hand right, and sometimes it doesn't. And some days it comes off your hand right more consistently than others. I don't see how you can believe in the importance of technique and fundamentals and at the same time not believe in the hot hand. It's not something magical. The hot hand is about being in good form. At least for me it is.

[sagegrouse]

I can take the string of 3 point shots for any player in his career and tell you whether they are independent or not. There is no variable to account for there...but as someone with domain knowledge, that number would be meaningless

It all comes down to the interpretation. Do you count shots across games? Do you count across halves? Do you account for how closely guarded the player is? I mean maybe that is accounting for variables.

But the point is the calculation itself is not accounting for anything. It all comes down to whether you think that calculation yields relevant data, and as you say, domain knowledge helps you identify that...and then only really bother putting weight in the calculations that DO yield relevant data.

You need both...you need stats, you need basketball people. It could be that all shots are independent, and that result could be meaningless. It bothers me a lot when people think there is this massive divide between stats and the "eye test". You need both...and you will be woefully incomplete without either.

As we regularly demonstrate here, often times people's impressions end up very wrong...but we also find that stats can't be trusted blindly...what stats do you even know are relevant without domain knowledge? As I pointed with the pants anecdote...without domain knowledge I would have thought no-pants == wins. I wouldn't say that was a variable that wasn't accounted for...but a stupid thing to look at in the first place! I can still say that at the 99.6% confidence level, pants have an effect...and I wouldn't be wrong.

The defense against such ridiculous results as your example is usually the "power of the test." Mining a data base looking for high correlations will often produce results that appear to be statistically significant, but which are truthfully meaningless (such as your example). This approach almost guarantees that the probability of accepting a false hypothesis is very, very high, and, therefore, the "power of the test" is correspondingly low.

Kindly,
Sage
'Sometimes I really think I know what I am talking about, but surely no one else is fooled'

[killerleft]

Having the "hot hand" is as real as it gets. Without going into boring basketball anecdotes, I got "hot" (as we usually called it) many a time on the playgrounds. I often wondered how and why it happened, but I could never really quantify it.

Trying to account for the hot hand mathematically is futile, in my opinion. Looking back, I would say that what getting hot boiled down to was 1) a high level of confidence 2) recognizing that "feeling" from earlier play, and 3) acting upon it with joyous abandon. Long time ago, but there was sort of a feeling of "deja vu", at least the weirdness factor part of it. If I started feeling it, I resorted to the old "give and go" offense, which translates into "give me the ball and go to h___."

I never, ever got "hot" playing golf or softball. Had some good games, but I guess the rythym of those sports is too slow. Hey, wait, add rythym to the equation.

[MChambers]

This discussion reminds me of a quote I recently saw from someone named Jon Ronson:

"Ever since I first learned about confirmation bias, I've been seeing it everywhere."

https://books.google.com/books?id=iR...ywhere&f=false

Jokes aside, I found the Slate article a little confusing. Guess I'll have to read the underlying study. It amazes me that Amos Tversky could make the mathematical error claimed in the article, but anything is possible.