Lies, Damned Lies, and Statistics...

[DukieinSoCal]

KenPom was a win probability graph for the game, but it's behind his paywall. Because of that I won't post it here but here is some key info. Our lowest probability to win was 13.7% at the 13:25 mark of the 2nd half when were down 48-39. Our win % at the start of the game was 45.1%.

I wonder what factors KenPom uses to come up with these numbers. I'm sure he somehow factors in Wisconsin's historically efficient offense but how do you account for the fact that they had upperclassman leadership that had been to the Final Four the previous year and the fact that we were in serious foul trouble with Jah and Justise? To me, it felt like our chances of coming back to win in this particular game and situation were very, very low. Hence, the amazing euphoria that overtook me as we stormed back to take home the natty. I'm still on could nine!

[Kedsy]

To me, it felt like our chances of coming back to win in this particular game and situation were very, very low.

I was at the game, and I wasn't particularly worried at all. I felt that if we had an empty possession and they hit a bucket to go up 11 or 12 then I'd start to worry. Instead, Grayson hit a three and it was a 6-point game. Like I said earlier, all we needed (down 9) was three-stop-three and it was a one-possession game. And you see three-stop-three all the time, right? It didn't occur to me until this discussion how unlikely that was.

So I guess different folks feel differently in situations like that.

[uh_no]

d. The evidence for the ACC is how well the conference did in the NCAA's: an unprecedented five teams in the Sweet Sixteen

Minor quibble...this statement is false
2009 Big east:

16: uconn, UL, Pitt, Nova, Cuse
8: uconn, Ul, Pitt, Nova
4: Uconn, Nova

NOthing against the rest of it. I thought far and away that the ACC was the best conference, having NCSU, UL, Duke, ND, and UNC all going as far as they did (big miss on UVA...but what are you going to do!). There is much too much evidence to not consider the ACC as far and away the best at the top this year.

[ice-9]

Why does the fact that a team with a fairly low probability of winning the tournament actually won it make the computer systems that determine the probabilities "faulty"? In three of the past five seasons, the team that won the tournament wasn't among the five most likely teams to win. And you think that means the rating systems are wrong? To me, it just means that the most likely team very often doesn't make it through a tough, one-and-done tournament. Frankly, the best team from the regular season rarely wins the NCAA tournament.

Also, the computer probability of winning it all depends very strongly on how difficult a path you have. Duke's percentage was low in large part because we had a 5-seed in our path (Utah) that the computers profiled as more of a 3-seed or even a 2-seed. Pomeroy, for example, ranked Utah as the #8 team in the country, and Gonzaga #6. Having to go through two top 10 teams is something none of the other #1 or #2 seeds had to achieve to get to the Final Four, hence Duke's chance of getting that far was lower than the other contenders. That makes perfect sense to me, not faulty at all.

I do think that this year, KenPom either greatly underestimated Duke or overestimated Duke's opponents in the NCAA tournament. I haven't looked up numbers, but my hunch is that Duke outperformed the predicted score in most of their games in the tournament.

The reason why Duke's initial odds were so low is because Utah and Gonzaga were seen as so good. San Diego State wasn't too bad either. Now that we've played the games, I think we can all agree Duke was clearly better than those two -- much better than what KenPom would have predicted.

[uh_no]

I do think that this year, KenPom either greatly underestimated Duke or overestimated Duke's opponents in the NCAA tournament. I haven't looked up numbers, but my hunch is that Duke outperformed the predicted score in most of their games in the tournament.

The reason why Duke's initial odds were so low is because Utah and Gonzaga were seen as so good. San Diego State wasn't too bad either. Now that we've played the games, I think we can all agree Duke was clearly better than those two -- much better than what KenPom would have predicted.

and more importantly because duke's tournament defense far outstripped almost anything it had done in the regular season.

I do think utah/zaga might have been slightly overrated in kenpom...but i would have had them top 15

[sagegrouse]

I do think that this year, KenPom either greatly underestimated Duke or overestimated Duke's opponents in the NCAA tournament. I haven't looked up numbers, but my hunch is that Duke outperformed the predicted score in most of their games in the tournament.

The reason why Duke's initial odds were so low is because Utah and Gonzaga were seen as so good. San Diego State wasn't too bad either. Now that we've played the games, I think we can all agree Duke was clearly better than those two -- much better than what KenPom would have predicted.

The ACC was systematically underrated due to a fairly mundane conference performance in November and December. In fact, the ACC was very, very good this year.

And also, Utah and Gonzaga were probably overrated: West Coast teams did not play many games against teams in the other power conferences.

[bedeviled]

i don't think there is any mathematical difference between
A. team x has a 55% chance of beating team y
B. team x will beat team y and that statement has a 55% chance of being correct

Perhaps you are calling attention to something I misstated. If you are speaking more generally, though, consider that the problem is that the model's statements and validations (B) are based on groups, not the singular game (A) in question. Thus, it should read:
A. team x has a 55% chance of beating team y
B. a team like x beats a team like y, and that statement has been true 55% of the time when looking at data from a group of such teams.
Unless the model has accounted for every possible factor, it cannot jump from point B to point A. There is a difference.

Declaring group odds (probabilities from a set of modeled/historical games) as if they were the odds of a singular event is a type of false equivalency unless all variables are equal. Moreover, the odds are also somewhat arbitrary if not all variables are factored in. That is, the odds depend on what criteria are used in the model. Different models will give different odds.

For example, say you have a history of flipping the following 4 coins and recording their outcomes. These represent your databank of games.

Denomination	Weighted?	Odds of Heads
Quarter	Yes	100%
Quarter	No	50%
Nickel	Yes	25%
Penny	Yes	0%

Now, you have a new coin (Duke vs Wisconsin), and you want to predict the odds of tossing a heads (Duke wins). The new coin is a weighted quarter. But, you don't know how it is weighted until you toss it...at which point you would find out it comes up heads 100% of the time. So, what are your pre-toss predictions? Well, it depends on what model you are using - various models think different things are important about coins, and arrange their data accordingly.

Model	Criterion Important to the Model	Silly Analogous Bball Criterion	Odds of Heads if Compared to Similar Coins from Databank
Model 1	"Coins are not distinguishable"	Cuban says, "MBB is an ish-show. Who cares?"	44%
Model 2	"Weighted coins are special"	Teams from Big12 win	42%
Model 3	"I've seen silver coins win more in the past. Silver is for winners!"	Teams with high Efficiency Margins win	58%
Model 4	"Quarters are worth more"	Teams w/ 5+ McD's	75%
		Qtrpounder All-Americans
Model 5	"This is a weighted, silver quarter"	2015 Duke reminds me of 1991	100%

So, we see that the stated prediction varies with whatever the model thinks is important, while the true odds remain constant. And, obviously, the stated predictions and the true probability are not the same...except for Model 5 which both considers all the variables in our simple example and includes an event exactly like our unknown event.

Hopefully, anyone who has a loved one with a tragic disease is aware that the group odds and the singular odds are not equal. Indeed, medicine is always working to refine their groupings/comparisons to give more accurate predictions.
(identifying biomarkers to separate out which patients will respond to a certain medication)
(better characterizing a disease to differentiate who has a progressive form vs protracted form)

A logical question, then, is 'How do we better refine our tournament basketball prediction?'
Because I find it interesting, I'd like to include one consideration for refinement. I have previously linked an Ohio State term paper on developing a model specifically for predicting the tournament (data was from 1986-2009 and tested on 2010,11). In developing her model, the author disovered that "% of seasons coach has gone to the Final Four" was a significant factor in predicting tourney results. I don't think it's a great leap to think that it would have been a factor in predicting 2015 results, too! Regarding the variable, she had this to say,

The very small number of coaches that this variable is meaningful makes it almost useless, but does imply that it is worth recruiting, and paying for, coaches who have made it to the Final Four and/or Championships.

For future considerations, I wonder if "Coach previously played in or was Assistant Coach in Final Four" would also be a significant factor.

[bedeviled]

I just went to the Kaggle March Madness competition site to see how those crazy smart folks did. Per Wikipedia,

Kaggle is a platform for predictive modelling and analytics competitions on which companies and researchers post their data and statisticians and data miners from all over the world compete to produce the best models

In one of the first posts in their forum, someone was bragging about being top 1% of the ESPN bracket pool...just before he mentioned the news that a 12-year-old ties for 1st in ESPN bracket challenge.

[Duke95]

The ACC was systematically underrated due to a fairly mundane conference performance in November and December. In fact, the ACC was very, very good this year.

And also, Utah and Gonzaga were probably overrated: West Coast teams did not play many games against teams in the other power conferences.

I would agree somewhat on Gonzaga. However, if you remember, quite a few people picked Gonzaga to beat us. With regard to Utah, I've seen them play a bit. They played Arizona tough. With Kansas, they had horrible first half jitters, then stormed back. They were a very strong team. Honestly, I would say they were slightly underrated.

[subzero02]

I would agree somewhat on Gonzaga. However, if you remember, quite a few people picked Gonzaga to beat us. With regard to Utah, I've seen them play a bit. They played Arizona tough. With Kansas, they had horrible first half jitters, then stormed back. They were a very strong team. Honestly, I would say they were slightly underrated.

Utah was probably under-seeded but they were not underrated by several of the computer models.

[jv001]

The ACC was systematically underrated due to a fairly mundane conference performance in November and December. In fact, the ACC was very, very good this year.

And also, Utah and Gonzaga were probably overrated: West Coast teams did not play many games against teams in the other power conferences.

SEC (minus UK) and West coast teams very much over rated and Duke was under rated at the end of the season. Duke's defense picked up towards the end of the season and made Duke one of the nations two or three best teams. Matter of fact we'll never know how good Kentucky was because of their very soft SEC schedule. GoDuke!

[Kedsy]

Matter of fact we'll never know how good Kentucky was because of their very soft SEC schedule. GoDuke!

What I think is overrated is the weakness of Kentucky's schedule. KenPom's final numbers rank UK's schedule as the 31st best in the country.

[Jarhead]

What I think is overrated is the weakness of Kentucky's schedule. KenPom's final numbers rank UK's schedule as the 31st best in the country.

When was that last calculated?

[CDu]

What I think is overrated is the weakness of Kentucky's schedule. KenPom's final numbers rank UK's schedule as the 31st best in the country.

To be fair, that is partially inflated by facing West Virginia, Notre Dame, and Wisconsin to end the tournament. It also may reflect some overvaluing of the SEC.

UK certainly did play and wallop some good teams: Louisville and UNC for example. But on the whole I do think their schedule was pretty soft.

[bedeviled]

To be fair, that is partially inflated by facing West Virginia, Notre Dame, and Wisconsin to end the tournament. It also may reflect some overvaluing of the SEC. UK certainly did play and wallop some good teams: Louisville and UNC for example. But on the whole I do think their schedule was pretty soft.

Overvaluing of Texas, as well.

What I find overvalued is the BPI. I hated hearing ESPN talk about it this year, and will be even more disgruntled next year.
Duke had the 67th hardest SOS according to the BPI....despite a full third of our games (13) being against the top 20 teams in the BPI! With such an easy schedule, it's no wonder we were able to finish 6th, lol.

[Kedsy]

To be fair, that is partially inflated by facing West Virginia, Notre Dame, and Wisconsin to end the tournament. It also may reflect some overvaluing of the SEC.

UK certainly did play and wallop some good teams: Louisville and UNC for example. But on the whole I do think their schedule was pretty soft.

It was definitely inflated by UK's tournament games. But before the tournament started, Kentucky's schedule was still rated 41st by Pomeroy. Not as strong as Duke's (16th pre-tourney), but a statement saying UK was overrated because they played such a poor schedule wouldn't reasonably seem to be speaking about the 41st best schedule in the country.

[subzero02]

To be fair, that is partially inflated by facing West Virginia, Notre Dame, and Wisconsin to end the tournament. It also may reflect some overvaluing of the SEC.

UK certainly did play and wallop some good teams: Louisville and UNC for example. But on the whole I do think their schedule was pretty soft.

Although Kansas fans are probably happy that you didn't list them amongst the good teams that Kentucky stomped; rock chalk should probably be listed first after the beat down they received in the champions classic.( UK 72... KU 40)

[ice-9]

I would agree somewhat on Gonzaga. However, if you remember, quite a few people picked Gonzaga to beat us. With regard to Utah, I've seen them play a bit. They played Arizona tough. With Kansas, they had horrible first half jitters, then stormed back. They were a very strong team. Honestly, I would say they were slightly underrated.

Now that you've seen the games, in a series of 10, how many times do you think Gonzaga and Utah would win against Duke?

I'd put it at two, maybe three. Duke would win 80% of the time against those teams. Gonzaga and Utah aren't on Duke's level, despite what KenPom thinks.

(And really, maybe 90%. The more times they play against each other the more apparent the talent differential.)

[CDu]

Now that you've seen the games, in a series of 10, how many times do you think Gonzaga and Utah would win against Duke?

I'd put it at two, maybe three. Duke would win 80% of the time against those teams. Gonzaga and Utah aren't on Duke's level, despite what KenPom thinks.

(And really, maybe 90%. The more times they play against each other the more apparent the talent differential.)

The way Duke played in the tournament (most specifically, the way Duke played DEFENSE in the tournament), Pomeroy would have absolutely agreed with this sentiment. The "problem" (from a modelling perspective) is that Duke didn't play defense like that during the season. They showed flashes of it later in the season (most notably in the first half against Notre Dame in Cameron), but throughout the year they just seemed to do enough defensively to let their offense win by a comfortable margin.

Pomeroy (and all of the models) probably had Duke rated appropriately based on their body of work; they just didn't have a way of predicting that Duke's defense would come to life so brilliantly for 6 straight games in the NCAA tournament.

[Troublemaker]

Now that you've seen the games, in a series of 10, how many times do you think Gonzaga and Utah would win against Duke?

I'd put it at two, maybe three. Duke would win 80% of the time against those teams. Gonzaga and Utah aren't on Duke's level, despite what KenPom thinks.

(And really, maybe 90%. The more times they play against each other the more apparent the talent differential.)

I'm not sure the eye test is reliable in one-game samples.

If Duke would beat Miami 8 out of 10 times, but all you saw was the game in Cameron, you might think it was actually Miami that would beat Duke 8 out of 10.

And, maybe Duke really does only beat Utah 6 out of 10 times but because we were fortunate to witness one of the 6 the first and only time the two teams played, we now perhaps mistakenly believe Duke would win 8 out of 10.