NET News (January 28): Who would the No. 1 seeds be based on blind resumes?

[uh_no]

That sounds like a fascinating, and valid metric. I'm not too familiar with that...gonna Google that guy and learn more. I think "in control" - if his numbers capture it - is incredibly valuable and predictive information.

you definitely could attempt to regress teams efficiency vs incontrolledness, and begin to discount negative impacts of letting off the gas.

[HereBeforeCoachK]

Yeah, Hingeknocker got it right. Efficiency-based systems like Pomeroy and Torvik don't give any extra boost for winning the game.

That sounds like a little bit of a fly in the ointment there...how would you tweak those systems to include some kind of adjustment based on winning or losing...

[uh_no]

That sounds like a little bit of a fly in the ointment there...how would you tweak those systems to include some kind of adjustment based on winning or losing...

actually winning the game provides very little predictive value above efficiency.

if you want to give bonus points for winning, net does exactly that.

[HereBeforeCoachK]

actually winning the game provides very little predictive value above efficiency.

if you want to give bonus points for winning, net does exactly that.

Right, but it does, and should, provide a little extra....garbage time and all....

[House P]

I remember reading once where a guy talked about creating a rating system based on minute-by-minute win probabilities -- there are people who calculate the chance of each team winning at each minute of the game -- which would presumably take pace into account and would also eliminate the problem you're talking about, as well as the need for diminishing returns for larger point spreads. I've never actually seen such a rating system, though, so either (a) it was too difficult to create such a system that made sense; (b) the information necessary for such a system wasn't readily available; or (c) it's out there but I haven't seen it and it never got popular enough to matter.

In his T-Rank system, Bart Torvik does a couple of things to account for scenarios where the final couple minutes are not played like a "normal" game (e.g. when a large lead narrows during garbage time or when a single digit lead grows to double digits when the trailing team fouls out of desperation). Here is my understanding of how we addresses a couple specific scenarios.

1. Garbage time: Possession which occur after a lead has been determine to be 100% safe are significantly de-emphasized in T-Rank. The point at which a lead is considered 100% safe is based on a formula developed by Bill James which says that a 20 point lead is safe with ~5 minutes left and a 30 point lead is 100% safe with ~12 min left (anyone interested in more details can click this link). Using the Duke/Clemson game as an example, Duke's lead was considered 100% safe when Duke led by 28 with 9.5 minutes left. Therefore, the fact that Duke was outscored 23-14 for the rest of the game doesn't impact Duke's overall T-Rank rating very much.

2. Game Control: Instead of using only the final score as an input, T-Rank also uses the "average lead" over the course of the game. This can help differentiate a tight game which ended up with a 10 point margin due to fouling from a game where one team led by double digits most of the way, but the final margin ended up at 10 points. As Hingeknocker points out, T-Rank lists the average margin as "+/-" on each team page. Two good examples for Duke are the Texas Tech and Pitt games. The final margin was 11 vs Texas Tech and 15 vs Pitt, but the average margin was -0.8 for Texas Tech and 13 for Pitt. This reflects the fact that the Texas Tech game was significantly more competitive than the Pitt game.

3. Blow outs between mismatched teams: If the margin of victory is more than 10 points and the overall ranking between two teams is above a certain threshold, the result of the game starts getting discounted. . However, as an example, the influence of Duke/Stetson game is discounted by 75% compared to a "normal" game. I should point out that Duke's blowout of Kentucky is not discounted because the two teams are not considered to be "mismatched". Anyone interested in the specific calculation can check out this link.

I know that KenPom adjusts for blowouts, but I am not sure he makes the other adjustments.

Finally, while KenPom is still the go-to guy for college basketball analytics, I am growing to really like the (free) content T-Rank provides. I would highly encourage anyone interested in this kind of stuff to check out his site (and listen to the episode of Jordan Sperber's podcast where Bart Torvik was a guest ).

[Nugget]

Thanks! Really appreciate the clarity/detail of the explanation. And, agree that Sperber's podcast is very well done and illuminating.

[uh_no]

Right, but it does, and should, provide a little extra...garbage time and all...

NET, sure. Rewarding teams for actually winning is important.

KP is a predictor. w/l provides no marginal predictive value, and thus is not factored in. He had posts a while back where he did a study on clutch and close games, and clutchness and winning close games are not evidence of clutchness or winning close games moving forward.

[Ian]

I'm just glad we aren't using the RPI any more. Kansas is still #1 RPI even after their loss tonight.

[BandAlum83]

I'm just glad we aren't using the RPI any more. Kansas is still #1 RPI even after their loss tonight.

Seriously? Are you sure you don’t need to refresh you browser? I would have thought they would have dropped at least 3 losses ago!

That’s really bad.

[ElliottHoo]

Seriously? Are you sure you don’t need to refresh you browser? I would have thought they would have dropped at least 3 losses ago!

That’s really bad.

RPI really, really loved SoS and Kansas has that in spades.

[camion]

RPI really, really loved SoS and Kansas has that in spades.

I know I've said this before. My understanding of the RPI is that it was not designed to rank the teams in order of their own strength or likelihood of winning. It was created to discourage teams from loading their schedules with cupcakes. As such it was better than nothing.

[HereBeforeCoachK]

I know I've said this before. My understanding of the RPI is that it was not designed to rank the teams in order of their own strength or likelihood of winning. It was created to discourage teams from loading their schedules with cupcakes. As such it was better than nothing.

But the NCAA used it for the former, even as it was designed for the latter. That was the problem.

[camion]

But the NCAA used it for the former, even as it was designed for the latter. That was the problem.

The NCAA used RPI as a tool in ranking teams. That doesn't mean that the ranking was necessarily an ordering of the teams' likelihood of winning. Part of that ranking could be how well a team conforms to the NCAA's scheduling wishes.



Note: I really don't enjoy defending the NCAA on any front, but I'm okay with this particular aberration.

[HereBeforeCoachK]

The NCAA used RPI as a tool in ranking teams. That doesn't mean that the ranking was necessarily an ordering of the teams' likelihood of winning. Part of that ranking could be how well a team conforms to the NCAA's scheduling wishes.



Note: I really don't enjoy defending the NCAA on any front, but I'm okay with this particular aberration.

So in your opinion, the RPI never led to any unfair/bad seedings?

[scottdude8]

Glad to see this discussion going strong! Sorry for the lack of a new post this week, but it’s vacation time, haha. Maybe next week I’ll try to do a RPI/NET comparison or something like that.

[uh_no]

So in your opinion, the RPI never led to any unfair/bad seedings?

i don't think he said that at all ...

[HereBeforeCoachK]

i don't think he said that at all ...

Actually, when you look at my post that he responded to...then looked at his...there is no other reason to make that reply unless that was the point. I don't think he really means that, but context is everything...and there was no reason to debate my point unless you are willing to stand by that. The context was that the RPI would from time to time lead to bad seedings...not all the time...so my burden of proof was low.

[Nugget]

I may have to listen to this whole section, because I actually think there's a reasonable compromise to be had between efficiency (NET, KenPom, etc) and results-based evaluations. Selecting the teams for the tournament should be based on a host of factors, including your black-and-white wins and losses, that efficiency metrics aren't designed to include. As a hypothetical, it would be possible for a team to go 10-20 on the season, but end up with a relatively high KenPom rating; perhaps they played a very hard schedule, and all of their 20 losses came by 1 point. But, obviously, this team should not make the tournament.

However, seeding the teams absolutely, without a doubt, should be done via the efficiency metrics. The predictive metrics are all that matter once the tournament has its 68 teams. If you're a team who earns a 1 seed under this metric, you deserve to play a team truly of the caliber of an 8/9 seed in the 2nd round, 4/5 in the S16, etc. etc. I would be perfectly happy if the tournament seeds were a straight S-curve of the KenPom rankings at the end of the year (with the usual bracketing rules in place, of course).

All in all, very intelligent observation and conversation about the systems. Thank you for the recap!

To your point, the interview Matt Norlander posted this morning with Committee Chair Bernard Muir at CBS Sports.com includes this from Muir indicating that they see the predictive metrics as having specific value related to seeding:

"Muir: I would say I'm happy with our process. With how this NET plays out, what we glean from it this year will be interesting. It's been helpful the past couple of years as we've introduced predictive metrics in our discussion. More so I think that's been a good thing for us especially in terms of seeding and getting that seeding right because that's so important to creating a balanced bracket and a great national tournament. I..."

[BandAlum83]

RPI really, really loved SoS and Kansas has that in spades.

What good is SoS if you lose the games?

[HereBeforeCoachK]

What good is SoS if you lose the games?

That does seem like the logical question, doesn't it?