uh_no
01-10-2016, 04:42 PM
I'm growing tired of the subjective arguments happening in all the women's basketball threads. Instead of complaining, lets do something about it.
I will be the first to claim that I've made far more assumptions here than I generally would like, but ease of access to data sometimes necessitates corner cutting.
Ultimately what I want to show is to what degree different programs have translated recruiting success into tournament success. To do this, on a super high level, I came up with a metric for recruiting success, and a metric for tournament success and divided one by the other to get a final value.
Major assumption 1:
Teams generally have a consistent level of recruiting success. This means that 4 years of recruiting data can be used to assign a recruiting metric to a program going further back in time. Why 4 years? because that's all the data i could get before ESPN's ratings site fell apart. As there haven't been major coaching changes at programs aside from tennessee, I think this is a RELATIVELY safe assumption. Ideally, I would associate tournament performance for each year with only the recruiting classes for the past 4 years (perhaps with some sort of weighting for upperclassmen), but since I only have 4 years worth of recruiting data...oh well. I don't think it's a monumentally flawed assumption, maybe only a little flawed.
Major assumption 2:
over 4 years, bad tournament matchups, close losses, etc will even themselves out. It's probably not perfect, but it's not horrible either.
Major assumption 3:
tournament success translates to good program performance. This has generally been exceptionally true in the women's game, and is due to the relative disparity among teams at the top. Further, this particular analysis only purports to deal with tournament performance, so I'm fine here. If you find me convincing evidence that for women's teams, tournament performance does not highly correlate with regular season performance, I'd love to hear it. This is necessitated by the difficulty of objectively analyzing regular season performance given disparate league schedules. I considered trying to do something like how recruiting success determines the outcome of a game between two teams (which would be even better), but that would be hard to get the data for, and I suspect the results would be more or less the same.
Recruiting metric:
for each of the years, you got 25 points for a #1 class, 24 for #2, etc. unranked teams got 0 points. So 4 #1 classes gets 100 points.
Minor assumption: an unranked recruiting class is as good as the #25 class. I doubt this is that big a deal...we could do negatives, but i don't think it's worth it...as it's generally due to only having 1 good player in a class, and that team probably had good classes surrounding it. Further, once you get down in the rankings, the difference between rankings is probably pretty small...so 0 points it is.
Minor assumption 2: the difference between 1 and 2 is the same as between 2 and 3...etc for all ranked classes. This probably ISN'T strictly true, but it's not so far off as to concern me
Tournament metric:
Each win is worth 1 point. the result is scaled so a championship title is 25 points.
Teams analyzed:
I chose the teams that showed up at the top of the recruiting rankings often or performed well in the tournament. there's not great rhyme or reason for teams included.
Duke
ND
Tenn
Uconn
BU
UL
Stanford
USC
Maryland
Recruiting results
Team
Recruiting Rating
Duke
88
ND
82
Tenn
70
Uconn
68
Maryland
62
Baylor
51
UL
39
Stanford
32
USC
24
Tournament Results:
Team
Recruiting Rating
Uconn
92
ND
79
BU
58
Maryland
54
Stanford
50
Tenn
46
UL
46
Duke
38
USC
38
Combined Tournament Performance/Recruiting Performance
Team
Recruiting Rating
Stanford
158
USC
158
Uconn
134
Louisville
116
Baylor
114
Notre Dame
97
Maryland
88
Tennessee
66
Duke
43
Results:
* Duke has had the best recruiting per ESPN.
* Of the teams at the top of the recruiting rankings, Duke has had the least tournament success
* Duke is among the lowest performing tournament teams given their recruiting success. Unless there is some team which had a ranked class in the past 4 years and no tournament wins, I suspect Duke may have the lowest value in the country.
Conclusions:
Given this data, there are a few conclusions one can draw as it pertains to Duke Basketball
* ESPN's recruiting rankings are flawed. I actually believe this to be the case to some degree. I feel there is too much weight placed on class depth. 4 pretty good recruits can trump 1 elite recruit...and in real games, that one elite recruit can often beat your 4 pretty good recruits.
(griner, diggins, stewart, et. al)
* Perception of recruiting rankings is flawed based on how the recruits developed at the school. Was player X what made team Y good? or did team Y make player X good? If Elizabeth Williams and Stephanie Dolson had swapped places, would the new Duke or Uconn teams have won the titles?
* Duke fails to translate their recruited talent into on court success, perhaps more than any program in the country. If you put ANY weight on recruiting rankings and player development within a program, than to some degree, I don't see how you can NOT accept that Duke has done less with more.
* There may be some confounding factor (transfers, injuries) affecting tournament results in a particular year. I would argue 1) that transfers are part of running the program, and 2) injuries bite every team at some point. Duke is not the only team that has injuries, and given their recruiting, should have the depth to deal with them.
So there it is. I feel I've been honest about the shortcomings of the methodology, though I don't believe improvements in methodology would significantly change the results, especially for Duke (given, that's a bit of a circular argument...i don't think the results would change because the results match my intuition, and my intuition is reinforced by what the (perhaps flawed) results show). Some changes I would have strongly considered, such as diminishing returns in recruiting rankings (1-2 > 2-3) I didn't include because it would have made it even worse for duke.
I want to be perfectly clear that I began this experiment with the goal of being perfectly content to have results go contrary to my expectations, though I did expect the results to come out as they did. I further do not think my bias affected the results (but I'll leave that to others to judge...I feel like Harry in Methods of Rationality experimenting that magic is a geneological trait despite knowing how the results would turn out). This is not an excuse to be destructively negative to the players, the coaches, or anyone else associated with the program (and if you do, I will be the first to report you to the mods). It is simply a result.
I will be the first to claim that I've made far more assumptions here than I generally would like, but ease of access to data sometimes necessitates corner cutting.
Ultimately what I want to show is to what degree different programs have translated recruiting success into tournament success. To do this, on a super high level, I came up with a metric for recruiting success, and a metric for tournament success and divided one by the other to get a final value.
Major assumption 1:
Teams generally have a consistent level of recruiting success. This means that 4 years of recruiting data can be used to assign a recruiting metric to a program going further back in time. Why 4 years? because that's all the data i could get before ESPN's ratings site fell apart. As there haven't been major coaching changes at programs aside from tennessee, I think this is a RELATIVELY safe assumption. Ideally, I would associate tournament performance for each year with only the recruiting classes for the past 4 years (perhaps with some sort of weighting for upperclassmen), but since I only have 4 years worth of recruiting data...oh well. I don't think it's a monumentally flawed assumption, maybe only a little flawed.
Major assumption 2:
over 4 years, bad tournament matchups, close losses, etc will even themselves out. It's probably not perfect, but it's not horrible either.
Major assumption 3:
tournament success translates to good program performance. This has generally been exceptionally true in the women's game, and is due to the relative disparity among teams at the top. Further, this particular analysis only purports to deal with tournament performance, so I'm fine here. If you find me convincing evidence that for women's teams, tournament performance does not highly correlate with regular season performance, I'd love to hear it. This is necessitated by the difficulty of objectively analyzing regular season performance given disparate league schedules. I considered trying to do something like how recruiting success determines the outcome of a game between two teams (which would be even better), but that would be hard to get the data for, and I suspect the results would be more or less the same.
Recruiting metric:
for each of the years, you got 25 points for a #1 class, 24 for #2, etc. unranked teams got 0 points. So 4 #1 classes gets 100 points.
Minor assumption: an unranked recruiting class is as good as the #25 class. I doubt this is that big a deal...we could do negatives, but i don't think it's worth it...as it's generally due to only having 1 good player in a class, and that team probably had good classes surrounding it. Further, once you get down in the rankings, the difference between rankings is probably pretty small...so 0 points it is.
Minor assumption 2: the difference between 1 and 2 is the same as between 2 and 3...etc for all ranked classes. This probably ISN'T strictly true, but it's not so far off as to concern me
Tournament metric:
Each win is worth 1 point. the result is scaled so a championship title is 25 points.
Teams analyzed:
I chose the teams that showed up at the top of the recruiting rankings often or performed well in the tournament. there's not great rhyme or reason for teams included.
Duke
ND
Tenn
Uconn
BU
UL
Stanford
USC
Maryland
Recruiting results
Team
Recruiting Rating
Duke
88
ND
82
Tenn
70
Uconn
68
Maryland
62
Baylor
51
UL
39
Stanford
32
USC
24
Tournament Results:
Team
Recruiting Rating
Uconn
92
ND
79
BU
58
Maryland
54
Stanford
50
Tenn
46
UL
46
Duke
38
USC
38
Combined Tournament Performance/Recruiting Performance
Team
Recruiting Rating
Stanford
158
USC
158
Uconn
134
Louisville
116
Baylor
114
Notre Dame
97
Maryland
88
Tennessee
66
Duke
43
Results:
* Duke has had the best recruiting per ESPN.
* Of the teams at the top of the recruiting rankings, Duke has had the least tournament success
* Duke is among the lowest performing tournament teams given their recruiting success. Unless there is some team which had a ranked class in the past 4 years and no tournament wins, I suspect Duke may have the lowest value in the country.
Conclusions:
Given this data, there are a few conclusions one can draw as it pertains to Duke Basketball
* ESPN's recruiting rankings are flawed. I actually believe this to be the case to some degree. I feel there is too much weight placed on class depth. 4 pretty good recruits can trump 1 elite recruit...and in real games, that one elite recruit can often beat your 4 pretty good recruits.
(griner, diggins, stewart, et. al)
* Perception of recruiting rankings is flawed based on how the recruits developed at the school. Was player X what made team Y good? or did team Y make player X good? If Elizabeth Williams and Stephanie Dolson had swapped places, would the new Duke or Uconn teams have won the titles?
* Duke fails to translate their recruited talent into on court success, perhaps more than any program in the country. If you put ANY weight on recruiting rankings and player development within a program, than to some degree, I don't see how you can NOT accept that Duke has done less with more.
* There may be some confounding factor (transfers, injuries) affecting tournament results in a particular year. I would argue 1) that transfers are part of running the program, and 2) injuries bite every team at some point. Duke is not the only team that has injuries, and given their recruiting, should have the depth to deal with them.
So there it is. I feel I've been honest about the shortcomings of the methodology, though I don't believe improvements in methodology would significantly change the results, especially for Duke (given, that's a bit of a circular argument...i don't think the results would change because the results match my intuition, and my intuition is reinforced by what the (perhaps flawed) results show). Some changes I would have strongly considered, such as diminishing returns in recruiting rankings (1-2 > 2-3) I didn't include because it would have made it even worse for duke.
I want to be perfectly clear that I began this experiment with the goal of being perfectly content to have results go contrary to my expectations, though I did expect the results to come out as they did. I further do not think my bias affected the results (but I'll leave that to others to judge...I feel like Harry in Methods of Rationality experimenting that magic is a geneological trait despite knowing how the results would turn out). This is not an excuse to be destructively negative to the players, the coaches, or anyone else associated with the program (and if you do, I will be the first to report you to the mods). It is simply a result.