I complained last week that the NCAA hadn't released NET (the replacement for RPI) yet. And apparently they listened to me ;-)
Here are the first NET rankings (which apparently will be updated daily):
https://www.ncaa.com/rankings/basket...l-net-rankings
Early season results are going to be wonky, of course.
https://pbs.twimg.com/media/Ds8ejgRWoAIhxX7.jpg
I'd love to know what metrics they are using. Kansas has beaten good teams and yet cannot even crack the top 10. Virginia has played one of the worst schedules in all of D1, embarrassingly weak, and yet they are the #2 team. Loyola Marymount is the #122 KenPom team with the #314th weakest schedule... on what planet does any metric make them the #10 team in the country?
I'm hoping this is just early season wonk and not some indication of a really fundamentally flawed formula.
Google claims that AI algorithms are used in the formula. Maybe our robot overlords have seen the future and this is what it holds. Could someone call John Connor? Seriously, this poll is nonsensical. If there’s insufficient data to apply the formula then they should wait another month before releasing it.
A computer ranking system that overvalues UVA and Wisconsin?! Get right outta town!!! /sQuote:
Originally Posted by JasonEvans
Funny little aside--KenPom also has a POY calculator/predictor, and when Zion was boasting a PER of 50+, the algorithm still had Ethan Happ as the favorite for POY based on early season metrics. Grant Williams (Tenn) was second. I imagine that's changed in one or many ways since I last looked, but thought it was funny. Maybe not 'haha' funny, but 'Zach Galifianakis doing math in his head' funny. It's pretty useless to look at the analytics-based rankings this early in the season, but useless =/= without some entertainment value.
NC State is 31st with the easiest schedule in the world...what's not to like about these new metrics ;)
Thanks for starting this thread Trouble, I was about to do it myself ;)
My initial reaction to this metric is that, while still (obviously) imperfect, it's a huge step forward over RPI, which was archaic at best. SI has an article I found explaining the NET rankings in a bit of detail: https://www.si.com/college-basketbal...system-explain
To me the biggest issue/unknown is how that "Team Value Index" is calculated... it definitely takes into account a variety of factors that I think are good, it just doesn't explicitly say what formula is used to get the final index. I'm also curious as to how the five factors are weighted, which is another detail that wasn't presented there. Perhaps there's a more detailed explanation somewhere else on the interwebs.
But again, this is a big improvement over RPI. The fact that the home/road factor is taken into account in two of the five factors makes me happy (I ranted and raved about the importance of that in evaluating teams for seeding a lot last year, haha). If it were me I might have eliminated scoring margin as it's own individual factor and instead made it a part of the "Team Value Index", because without context the margin of victory can be misleading (consider an 8 point margin that came about because the opposition fouled for the last minute and couldn't score, turning a close game into a larger margin, versus an 8 point margin that came about due to some late buckets with backups in the game). It also seems a bit silly to me to have both a pure and adjusted win percentage included. But maybe how the five factors are differentially weighted takes that into account.
A spork from my end to the first person who can find either A) the direct formula used for the "Team Value Index" or B) any clear explanation as to how each of the five factors are weighted, haha.
It seems you're not the only one chagrined at the missing formula(s)--here's an article from earlier today by Jerry Palm of CBSSports (and stoic champ of the RPI): https://www.cbssports.com/college-ba...mula-a-secret/. He specifically cites the very complaints that you mention (ambiguity of TVI, unclear weighting of factors, and seeming redundancy of some factors with others).Quote:
Originally Posted by scottdude8
Also, KenPom wrote something for The Atlantic in August breaking down NET, but I'm not a subscriber, so I can't speak to whether he was privy to the innards of the tool.Quote:
However, the thing that is missing from the what the NCAA revealed with the release of the NET ratings is any of the supporting data that goes into the rankings. All we are being given for now is each team's overall record and breakdown by home/road/neutral/non-Division I. That's it. There is not one piece of useful information on the rankings page except the ranking itself...
The NCAA has been working very hard in recent years to make the selection process as transparent as possible. Not releasing the formula for this is a significant step backwards in that process. And, while it may be so complicated that nobody on the actual committee can understand it, let alone explain it, surely there is one geek at every school that would. The geek writing this column is confident he would ultimately understand it also.
Here's what the NCAA itself had to say, when describing "everything you need to know about NET" (spoiler alert: it doesn't answer the questions posed): https://twitter.com/marchmadness/sta...etball-ranking
Apparently a game @Wofford is weightier than a neutral tilt with UK, the Zags, Auburn, etc. Who knew? (dare I say garbage in, garbage out?)
The CHeats are ranked #21 in the NET, which is significantly lower than the human polls. Are you looking at some sort of SOS component?Quote:
Originally Posted by HereBeforeCoachK
It may be an improvement over RPI, but almost anything would be. Pitt at #23? You gotta be kidding. tOSU at #1?
give it a break. any ranking system is going to be highly biased by preseason rankings at this point in the year. Even kenpom says his own numbers are a bit of a crapshoot until the end of january or so...and I can guarantee the NCAA puts less thought into the veracity of their preseason rankings than does he.Quote:
Originally Posted by MChambers
We'll see what it looks like in a couple of months.
Nate Silver is not impressed
"These are the worst rankings I've ever seen in any sport, ever. NCAA needs to go completely back to the drawing board...I guess I'm not sympathetic because a lot of smart people have worked on this problem (power rankings) for a LONG time and the NCAA ignored all that and came up with something that doesn't reflect methodological best practices and which doesn't make sense, basketball-wise"
Yes, I get the point, at least about small sample size. I assume NET doesn't have preseason rankings, though. Still, Pitt's schedule has been very weak.Quote:
Originally Posted by uh_no
My guess is that NET, like RPI, overvalues wins and undervalues scoring margin.
I stopped reading the thread after your post. This poll is beyond ridiculous right now... I'll take another look at it in a few weeks but right now it is pure crop.Quote:
Originally Posted by UrinalCake
Highly biased by preseason rankings? Ohio State was unranked in the AP in the preseason. They are now 6-0, with wins over Cincinnati, Creighton, and 4 cupcakes. And that team is ranked number one?Quote:
Originally Posted by uh_no
There must be a reason why the NCAA isn't releasing the details of this metric. Its performance looks even worse than RPI at this point.
This is a pretty harsh damnation by Silver, who is in a position to have his opinion taken seriously. Clearly these rankings are absurd. If Duke is only 6th after being 5-1 and beating Kentucky, Auburn and San Diego and barely losing to Gonzaga, then that's a system that will never ever have Duke a top seed. I thought these rankings were foolish before reading Silver's commentary - and afterwards, I'm even more convinced. Sounds like Silver kind of saw this coming.Quote:
Originally Posted by nmduke2001
Nate Silver, if he were wise, would be lauding why human analysis of data analytics is important and justifying his expertise as better than ML/“AI”.Quote:
Originally Posted by nmduke2001
Highly biased by preseason rankings? Not only were they unranked, Pitt was (is?) considered a laughingstock and at least a 3 year rebuild by Coach Capel. They are now 6-0, with wins over pre-season A-10 favorite St. Louis in Brooklyn (2X bonus points!), plus 5 cupcakes. And that team is ranked number 23?Quote:
Originally Posted by rasputin
There is a reason why the NCAA isn't releasing the details of this metric: student privacy rights, corporate partner contractual obligations, or the price of bagels in Oklahoma, take your pick. Those clowns wouldn't release their Chinese takeout order for lunch without a subpoena.
This silliness is the same thing as April statistics in baseball. "If Francisco Cervelli continues at his current pace, he would hit .444 with 80 home runs and 200 RBI." Have fun with it, and enjoy it while it lasts!
I'll leave the poll watching to the many stats geeks on this forum. I'll stick with actually watching the games.
Agreed that, to the limited extent they have identified the factors that compromise the NET (Net Efficiency -- which is just the raw points per possession on O less raw PPP on D; Winning Percentage, Winning Percentage adjusted by the factors in the RPI - i.e. giving a little bonus for neutral wins and a bigger bonus for road wins; Scoring Margin, capped at 10 points per game; and the mysterious Team Value Index -- which is described as having as components Wins and losses, game location and an "opponents" factor of unstated derivation, determination or weighting) it does not appear that any of them -- with the possible exception of the undefined "opponents" factor of Team Value Index -- is in any way affected by preseason rankings or pre-season expectations based on past performance (as for example KenPom's ratings are). So, this would all seem to be based solely on the limited sample size of the 5-7 games teams have played to date.Quote:
Originally Posted by Turk
And, clearly, it is heavily skewed towards simply winning games and winning road games -- so, for example, while Ohio St. probably isn't #1 they do have two very good wins in true road games at Cincy and at Creighton that would help them with the small sample size to date.
Also, it isn't quite right that the NCAA would never release the details behind their metrics -- they did release the formula for how the RPI was determined, which was one reason it was so easy to ridicule.
While I agree with those who've cautioned it doesn't make sense to get too worked up over the wonkiness of these initial ratings in light of the small sample, I absolutely agree with Palm and others who've urged the NCAA to release the forumulas/weightings so that the dorks who could do the necessary checks and balances to highlight errors/inefficiences can have enough to work with to suggest improvements.