KenPom and the Dorks 2022-23

[Dr. Maturin]

Multiple things can be true. Yes, the ACC really was down last year. Some of the results prior to January were bad. In particular, UVA and UNC were much worse than usual during that period. That set the narrative of a league with one good team and a bunch of squads just hoping to make the tournament.

It's also true that some real improvement happened in ways that wasn't totally captured by the analytics. It seems to be that there isn't much that can raise the profile of a conference once league play starts. I suppose that is an unintended consequence of the 20-game schedule.

[rsvman]

Well, sure. I'll give you your hypothetical exercise. But I don't understand your bigger point...
.

Pretty sure his point was that the conference in all likelihood got better sometime between November and March, and that, in retrospect, it is quite difficult to say at what point in time it got better when all you know is 'bad in November, good in March.'

In other words, could it have just been a string of good games and luck that only happened during the tournament? Possibly, but pretty unlikely, in my opinion. Could we have suddenly gotten better in December? Possibly, but again pretty unlikely.

So it seems that the most likely explanation is that the conference got progressively better over the course of those few months, but that it got dismissed or ignored based on how badly we fared in November.

[Bluedog]

Several years ago Pomeroy did an analysis comparing preseason rankings to end of season rankings as predictors of NCAA Tourney performance. So on one side we had rankings based on 0 games, just based on talent of incoming and returning players, program/coach assessments, years of experience, etc. On the other side we have a full season of play data for every team.

Tested over several seasons which was the best predictor of how far a team went in the NCAA’s?

Yep, you guessed it — preseason rankings!

That was an *amazing* finding imo. It argues that much of the season data is noise and that when crunch time comes talent, programs & coaches are what count most.

I would like to see the study replicated with recent year data, where there is more turnover. Preseason rankings might do even better since talent takes a front seat when you have less experienced teams? Ir it could go the other way - need to see the younguns
play 25+ games to really assess them.

I believe this was in respect to the preseason AP poll rankings, NOT preseason "computer rankings". I'd argue it's harder for the computers to update their algorithms in the high turnover environment whereas humans actually are probably more adept at assessing returnees vs transfers vs new freshman, etc. I was suggesting that the preseason computer rankings are probably less accurate than they used to be because of the challenges presented by turnover (and algorithms weighing all those attributes in the appropriate manner) although I certainly haven't done a robust analysis of it (or done the analysis compared to preseason AP polls). But my reference above was to computer preseason rankings, not human polls.

But you're right that the finding is certainly an interesting one -- 538 similarly takes preseason poll rankings into account when making NCAA predictions.

[Skydog]

I believe this was in respect to the preseason AP poll rankings, NOT preseason "computer rankings". I'd argue it's harder for the computers to update their algorithms in the high turnover environment whereas humans actually are probably more adept at assessing returnees vs transfers vs new freshman, etc. I was suggesting that the preseason computer rankings are probably less accurate than they used to be because of the challenges presented by turnover (and algorithms weighing all those attributes in the appropriate manner) although I certainly haven't done a robust analysis of it (or done the analysis compared to preseason AP polls). But my reference above was to computer preseason rankings, not human polls.

But you're right that the finding is certainly an interesting one -- 538 similarly takes preseason poll rankings into account when making NCAA predictions.

You are right - the focus was on AP rankings. And yes I believe Ken said he trusted AP (and other media) preseason rankings a bit more than his own preseason model. I don't remember if he commented on pre vs post season predictive power of his own computer rankings. Massey compares the predictive accuracy of all the ranking systems but I think the main conclusion from that is that no that there is a lot of movement in the accuracy rankings among the better polls. For example Kenpom (or Sagarin or x, or y) may be the 3rd most accurate one year and 12th the next. The crappy polls (ones that ignore margin of victory for example) seldom do well though. The volatility in "accuracy" among even the good polls makes sense though - they are being evaluated on the small sample size outcomes of the NCAA tourney.

As far as whether Kenpom type model preseason predictive power will decrease or increase given more player shuffling and churn, I can't really say. I still see arguments in both directions.

Btw 538 models these days seem to be very talent focused in their sports prediction models. IIRC their last season NBA model may have been entirely talent based, with regular season game outcomes counting almost nothing toward predicting playoff outcomes. I'm pretty suspicious of that approach. I've was a very big 538 fan in the past (and thought they actually did great in 2016, being maybe the only major non-partisan outlet to have Trump 30% to win the night before the election) but lately I think they have lost some of their rigor. That is just my impression though, based on reading about their models and frequent checks of their NBA and soccer predictions. Fairly or unfairly, Nate Silver's many lame tweets during the pandemic didn't help my perceptions of 538.

I'm still team Pomeroy all the way though.

[rsvman]

For the past 4 or 5 years I have printed out multiple copies of the NCAA tournament bracket and filled one out with Kenpom, one with Sagarin, one with RPI, and one with my own picks. So far, none of the systems has beat me.

I haven't actually compiled exact stats with regard to the others versus each other, but my impression off the top of my heed is that Sagarin is slightly better than Kenpom and that both are considerably better than RPI.

[Dr. Maturin]

The thing about the AP poll is that the preseason version is almost a different exercise altogether. During the season, teams get rated based on perceived results, with teams going up or down with a win or loss. To take one example, NC State played a poor game yesterday and was lucky not to lose. If the ECU kicker makes that field goal, they probably drop pretty far rankings. Since the field goal missed, they probably won't drop much.

The preseason poll is different. There, the voters actually try to rank the teams based on how good they are. I suppose it shouldn't be surprising that it has more predictive value.

[pcal6vb]

Regarding the ACC being down last year, vs being underrated: when the 2022 NCAA Tournament brackets were revealed, I remember asking myself (and some friends in a text chat) the question "Are teams 2 through 7 in the Big 10 standings and teams 4 thru 7 in SEC standings really that much better than teams 2 thru 7 in the ACC standings"? Part of this was my bias in watching all the ACC teams play Duke, but my gut had a hard time believing Illinois, Purdue, Iowa, Rutgers, Arkansas, A&M, LSU, and Florida were several seeds' worth better than unc, Notre Dame, Miami, Wake, UVA, and Va Tech.

I understand that going by the statistics (e.g. quality non-conference wins) the Big 10 and SEC teams above earned the higher seeds...but to the eye test, the teams above seemed relatively even. That perception seemed to largely play out in the results of the (admittedly high-variance, single elimination) tournament as well.

My takeaway is that the ACC was relatively on par, talent-wise and potential-wise, with the other conferences last year, at least in the middle of the standings. The middle ACC teams largely underachieved during the season, though, and therefore didn't earn the higher seedings that may have matched their talent/potential. The ACC also lacked 1 or 2 other stud teams at the top, along with Duke, that would have lifted the middle ("a rising tide floats all boats") the way that Auburn/Tennessee/Kentucky did in the SEC for example.

[Kedsy]

Regarding the ACC being down last year, vs being underrated: when the 2022 NCAA Tournament brackets were revealed, I remember asking myself (and some friends in a text chat) the question "Are teams 2 through 7 in the Big 10 standings and teams 4 thru 7 in SEC standings really that much better than teams 2 thru 7 in the ACC standings"? Part of this was my bias in watching all the ACC teams play Duke, but my gut had a hard time believing Illinois, Purdue, Iowa, Rutgers, Arkansas, A&M, LSU, and Florida were several seeds' worth better than unc, Notre Dame, Miami, Wake, UVA, and Va Tech.

I understand that going by the statistics (e.g. quality non-conference wins) the Big 10 and SEC teams above earned the higher seeds...but to the eye test, the teams above seemed relatively even. That perception seemed to largely play out in the results of the (admittedly high-variance, single elimination) tournament as well.

My takeaway is that the ACC was relatively on par, talent-wise and potential-wise, with the other conferences last year, at least in the middle of the standings. The middle ACC teams largely underachieved during the season, though, and therefore didn't earn the higher seedings that may have matched their talent/potential. The ACC also lacked 1 or 2 other stud teams at the top, along with Duke, that would have lifted the middle ("a rising tide floats all boats") the way that Auburn/Tennessee/Kentucky did in the SEC for example.

I would argue that it's difficult with one's eyes to tell the difference between, e.g., the #16 team in the country (i.e., the worst #4 seed) and the #29 team in the country (i.e., the best #8 seed). Before the tournament, Pomeroy rated Big 10 2nd-place team Illinois #17 (they got the last 4-seed) and ACC 2nd-place team UNC #29 (they got the first 8-seed). When you say those two teams didn't look so different, well if Illinois had played UNC on a neutral floor before the tournament, KenPom would have set the point spread as approximately 2.4 points. If you'd only seen each of those teams play a couple times, would you expect your eye to be able to discern such a small difference?

Personally, I would consider 2.4 points to be "relatively even" to my eyes, meaning your perception and the seeding could have both been correct.

The ACC's performance in the 2022 tournament was very similar to the Pac-12's performance in the 2021 tournament (best tournament record of any conference with three Elite Eight teams, after no Pac-12 team was seeded higher than #5). All during that season, the Pac-12 was considered to be one of the worst (if not the worst) of the major conferences. Was the Pac-12 underrated in 2021, or did a few Pac-12 teams just get hot in the post-season?

[flyingdutchdevil]

I would argue that it's difficult with one's eyes to tell the difference between, e.g., the #16 team in the country (i.e., the worst #4 seed) and the #29 team in the country (i.e., the best #8 seed). Before the tournament, Pomeroy rated Big 10 2nd-place team Illinois #17 (they got the last 4-seed) and ACC 2nd-place team UNC #29 (they got the first 8-seed). When you say those two teams didn't look so different, well if Illinois had played UNC on a neutral floor before the tournament, KenPom would have set the point spread as approximately 2.4 points. If you'd only seen each of those teams play a couple times, would you expect your eye to be able to discern such a small difference?

Personally, I would consider 2.4 points to be "relatively even" to my eyes, meaning your perception and the seeding could have both been correct.

The ACC's performance in the 2022 tournament was very similar to the Pac-12's performance in the 2021 tournament (best tournament record of any conference with three Elite Eight teams, after no Pac-12 team was seeded higher than #5). All during that season, the Pac-12 was considered to be one of the worst (if not the worst) of the major conferences. Was the Pac-12 underrated in 2021, or did a few Pac-12 teams just get hot in the post-season?

Both, just like the ACC in 2022. A little underrated (everyone crapped on the ACC last year, including me) and a few teams got hot (Duke had the talent, UNC had the experience, and Miami's backcourt finally clicked).

[uh_no]

Both, just like the ACC in 2022. A little underrated (everyone crapped on the ACC last year, including me) and a few teams got hot (Duke had the talent, UNC had the experience, and Miami's backcourt finally clicked).

miami also had an easy road. That's not there fault, but important in the context of evaluating their performance post-hoc.

[tommy]

I would argue that it's difficult with one's eyes to tell the difference between, e.g., the #16 team in the country (i.e., the worst #4 seed) and the #29 team in the country (i.e., the best #8 seed). Before the tournament, Pomeroy rated Big 10 2nd-place team Illinois #17 (they got the last 4-seed) and ACC 2nd-place team UNC #29 (they got the first 8-seed). When you say those two teams didn't look so different, well if Illinois had played UNC on a neutral floor before the tournament, KenPom would have set the point spread as approximately 2.4 points. If you'd only seen each of those teams play a couple times, would you expect your eye to be able to discern such a small difference?

Personally, I would consider 2.4 points to be "relatively even" to my eyes, meaning your perception and the seeding could have both been correct.

The ACC's performance in the 2022 tournament was very similar to the Pac-12's performance in the 2021 tournament (best tournament record of any conference with three Elite Eight teams, after no Pac-12 team was seeded higher than #5). All during that season, the Pac-12 was considered to be one of the worst (if not the worst) of the major conferences. Was the Pac-12 underrated in 2021, or did a few Pac-12 teams just get hot in the post-season?

I think that with respect to both the Pac-12's performance in 2020-21 and the ACC's in 2021-22, the sample sizes of the pre-conference games that determine much of the rankings and assessments, and the NCAA performances that are then used by some as "proof" of how good or bad a conference was, were quite small. A handful of games, really, both early season and post-season. So I think the reality is probably somewhere in the middle -- both leagues, in the years under discussion, were probably somewhat better than their performances in the pre-conference season suggested, and not as good as their post-season performances suggested.

[Billy Dat]

I never heard of Evan Miya until listening to a recent podcast, he's got an analytics-based college basketball blog that seems pretty cool and unique

https://evanmiya.com/

[DavidBenAkiva]

Erik Haslam of Haslametrics has released his 2022-23 ratings. Duke debuts at #9.

https://haslametrics.com/

FWIW, Duke debuted at #15 in last year's rating and moved up to #9 at the end of the season.

[CameronDuke]

2023 Kenpom preseason ratings are out.

3F391BF6-7FD1-4D61-95FD-DA864F7F7353.jpg

UVa 5, UNC-CH 9, Duke 15, VT 21.

[JasonEvans]

Virginia, the class of the ACC.

[DavidBenAkiva]

Ken Pomeroy talked about the poor success rate of 1st time head coaches on the DBR podcast, didn't he? He seems to be penalizing Duke based on Jon Scheyer. I think that's a fair assessment. Both Haslametrics and T-Rank, for example, have Duke in the top 10.

[UrinalCake]

I haven't seen this mentioned elsewhere, nor do I see a dork polls thread for this season, so I'll mention here that KenPom released his preseason rankings a few days ago. He forewarned during an appearance on the DBR podcast (which was awesome, btw) that Duke would be lower than we've been in recent years and that many Duke fans would be upset with him. Sure enough we checked in at #15, which to me actually sounds about right. As he explained in his own words, there's such a lack of data on our new roster plus the track record of first year coaches (based on limited data) isn't great. IMO it seems like Jon is in a great position to exceed expectations on that front given that most first year coaches are taking over a program that fired their previous coach because they weren't winning enough whereas Jon is starting with a roster loaded with talent, if not experience.

Anyways, it will be interesting to see how things play out. The bigger surprise to me is UNC-CHeat at #9. On paper they look like they have everything you could possibly want in a computer ranking - a team that made the title game and returned virtually their entire roster except for replacing one grad transfer for another highly regarded grad transfer. I'd be curious to hear why his formula has them so low. Perhaps it's because for most of last season they were actually a pretty bad team until they made their run (ugh) but even still it seems off.

https://kenpom.com/

[uh_no]

I haven't seen this mentioned elsewhere, nor do I see a dork polls thread for this season, so I'll mention here that KenPom released his preseason rankings a few days ago. He forewarned during an appearance on the DBR podcast (which was awesome, btw) that Duke would be lower than we've been in recent years and that many Duke fans would be upset with him. Sure enough we checked in at #15, which to me actually sounds about right. As he explained in his own words, there's such a lack of data on our new roster plus the track record of first year coaches (based on limited data) isn't great. IMO it seems like Jon is in a great position to exceed expectations on that front given that most first year coaches are taking over a program that fired their previous coach because they weren't winning enough whereas Jon is starting with a roster loaded with talent, if not experience.

Anyways, it will be interesting to see how things play out. The bigger surprise to me is UNC-CHeat at #9. On paper they look like they have everything you could possibly want in a computer ranking - a team that made the title game and returned virtually their entire roster except for replacing one grad transfer for another highly regarded grad transfer. I'd be curious to hear why his formula has them so low. Perhaps it's because for most of last season they were actually a pretty bad team until they made their run (ugh) but even still it seems off.

https://kenpom.com/

Not really. they weren't "all that good" overall last year, as far as computers goes. The fact that they made the title game doesn't undo the rest of the year that made it a shock they made the title game in the first place.

[DavidBenAkiva]

I haven't seen this mentioned elsewhere, nor do I see a dork polls thread for this season, so I'll mention here that KenPom released his preseason rankings a few days ago. He forewarned during an appearance on the DBR podcast (which was awesome, btw) that Duke would be lower than we've been in recent years and that many Duke fans would be upset with him. Sure enough we checked in at #15, which to me actually sounds about right. As he explained in his own words, there's such a lack of data on our new roster plus the track record of first year coaches (based on limited data) isn't great. IMO it seems like Jon is in a great position to exceed expectations on that front given that most first year coaches are taking over a program that fired their previous coach because they weren't winning enough whereas Jon is starting with a roster loaded with talent, if not experience.

Anyways, it will be interesting to see how things play out. The bigger surprise to me is UNC-CHeat at #9. On paper they look like they have everything you could possibly want in a computer ranking - a team that made the title game and returned virtually their entire roster except for replacing one grad transfer for another highly regarded grad transfer. I'd be curious to hear why his formula has them so low.

https://kenpom.com/

I am so glad the DBR guys had Ken Pomery on for his broadcast. When he did release his preseason rankings, I wasn't upset at all. I mean, I probably wouldn't have been upset anyway. It's just an algorithm and it changes over the season. But I had my expectations appropriately prepared for it.

As you note UrinalCake, UNC is quite low in the preseason KenPom as well. They don't appear in the top 3 of any of the released dork polls, as a matter of fact. T-Rank has them 4th and Haslametrics has them 6th. The preseason AP Poll is going to be released today. I assume they will be either #1 or #2 in that. Pretty much every other preseason ranking has them in the top 3. The analytics-based rankings are a little less enthusiastic about them.

Dork Poll thread for 2022-23 is here: https://forums.dukebasketballreport...-Polls-2022-23

[UrinalCake]

Thanks DBA, it looks like a mod graciously moved my post to the correct thread. I'm curious how strongly Pomeroy's preseason ranking weighs tournament performance from the previous season. If you treat all games as being equal then I agree they weren't very good. But recency bias alone (which I believe is a component of the formula) should give them more credit for their late run.