Page 1 of 2 12 LastLast
Results 1 to 20 of 40
  1. #1
    Join Date
    Feb 2007
    Location
    Steamboat Springs, CO

    The NCAA Is on a 'Fool's Errand' and Needs to Make Changes

    Over the past ten or more years, I have started a thread or made a post entitled, “The NCAA TSC Is on a Fool’s Errand,” referring, of course, to the lack of credible data to compare teams between conferences. The reason of course, if that almost all inter-conferences match-ups occur in the first month of the season, when teams are not at mid-season form and coaches have different approaches to bringing teams along.

    There have even been allegations that some conferences (Big 12, ahem) have intentionally scheduled and ran up the score on cupcakes to influence their overall conference rating and gain more money-yielding tournament selections.

    I would now go further and say the NCAA needs to make changes in its approach and give some credit to the past NCAA records of various conferences in its selections. And, if you say, that favors the big rich conferences, I would respond – “Well, not the SEC, whose record is 0.500 over the past three tournaments.” The Big Ten is scarcely better.

    And it is possible to take expected results for each seed into account. The good computer sci folks at the University of Illinois publish “Bracket Odds,” an updated historically based estimate of the number of wins for each seeded team, 1-16, plus the standard deviation of those wins. Thus, for each conference, based on the seed of its entries in March Madness, one can calculate wins and standard deviation of wins. (The latter is not a straightforward calculation but can be done.)

    For 2022, the five ACC teams were expected to win 5.17 games. Instead, the ACC won 14 games.

    In 2023, the five ACC teams were expected to win 5.22 games and instead won 7.

    In 2024, the five (again!) ACC teams were expected to win 7.34 games and instead won 12.

    The total for three years: expected wins – 17.7 wins. Actual wins – 33. The overall record was 33-15.

    The ACC over the past three years has greatly outperformed its NCAA seedings. The result of 33 wins would occur randomly only a few times out of 100,000 tries. (Tech. note – I converted to z-scores to more easily calculate overall std. dev.) To me, this means two things:

    (a) Our teams were seeded too low in the tournament. (I mean, ACC champ Duke as a #5 in 2023?)

    (b) Almost certainly, the ACC got cheated on the number of teams in tournament, costing big money to conference members.

    How did other conferences fare? Well, the woebegone Mountain West placed an eye-popping number of teams – six – (as Butch Cassidy exclaimed, “Who are those guys?!”). Their record this year was 4-6. The SEC was 8-8; the supposedly mighty Big 12 was 7-8. The Big East did well – with about the same number of teams as the ACC, they won 29 games. The Big Ten over the past three years has a record of 25-23. The SEC, Big Ten and Big 12 have averaged over seven selections per year. The ACC and Big East have average five.

    Surely there are questions. I can also calculate performance versus expectations for each of the other conferences, but it will take a day or two.
    Sage Grouse

    ---------------------------------------
    'When I got on the bus for my first road game at Duke, I saw that every player was carrying textbooks or laptops. I coached in the SEC for 25 years, and I had never seen that before, not even once.' - David Cutcliffe to Duke alumni in Washington, DC, June 2013

  2. #2
    Join Date
    Feb 2007
    Location
    NC
    I don't think this approach isn't any better than the current approach. In fact, I think it's worse in that it ignores current year evidence in favor of prior years. With teams changing year to year more than ever before, that's just not a tenable approach.

    All the conferences and all the teams know the metrics going into the season. It is on them to maximize their seeds accordingly. Is it a perfect process? No. But it is at least still based on the only relevant data, which are the teams' performances this year.

  3. #3
    Join Date
    Feb 2007
    Location
    Santa Cruz CA
    I could see something taking into account past performance coming in if there is an expansion to 96 teams.
    Back in the 70s when the tournament was smaller, I seem to remember there was a scheme in place to count past performances in regards to deciding which conferences got automatic bids. UNCC's FF appearance in 1977 made them an attractive team in that regard for a few years.

    As part of the negotiations to expand, possibly conferences get assigned multiple autobids based on past tournament performance.

  4. #4
    Join Date
    Feb 2007
    Location
    Washington DC
    Quote Originally Posted by CDu View Post
    I don't think this approach isn't any better than the current approach. In fact, I think it's worse in that it ignores current year evidence in favor of prior years. With teams changing year to year more than ever before, that's just not a tenable approach.

    All the conferences and all the teams know the metrics going into the season. It is on them to maximize their seeds accordingly. Is it a perfect process? No. But it is at least still based on the only relevant data, which are the teams' performances this year.
    While I mostly agree that using prior year's tournament data on current teams (especially in an era of significant player movement) would not necessarily solve the intended problem, I do think Sage's data shows that there may be systemic flaws in the current analytic framework that should be re-evaluated, modified, and back tested. So instead of just inflating the ACC, try to figure out if there is anything in the current model that is devaluing certain teams/conferences and see if those should be adjusted.

  5. #5
    Join Date
    Oct 2009
    Location
    Durham
    While i don't disagree with the thesis, the records are misleading. If the ACC and Big East had more teams, they'd necessarily have more losses, and likely records far more aligned with the SEC and B10. Might some of those teams have picked up a win? perhaps...but i think they'd probably have averaged ~1 win a team at best (and likely less than that). So the league record would look worse if those teams got in.
    April 1

  6. #6
    Unfortunately, I think it's on the ACC to play the system and perform better in early OOC play. But it's a double edged sword when you look at it on a team basis. Is it better for Duke to run up the score with its starters against a mid major to help raise the ACCs overall NET ranking? Or is it better for Duke to develop its deeper bench giving lots of minutes to players 8-10?

  7. #7
    Join Date
    Feb 2007
    Location
    Steamboat Springs, CO
    Quote Originally Posted by uh_no View Post
    While i don't disagree with the thesis, the records are misleading. If the ACC and Big East had more teams, they'd necessarily have more losses, and likely records far more aligned with the SEC and B10. Might some of those teams have picked up a win? perhaps...but i think they'd probably have averaged ~1 win a team at best (and likely less than that). So the league record would look worse if those teams got in.
    Which is my point. The bar was set too high. The gaudy records of the ACC and BE suggest they needed "higher seeds" and "more bids" -- and, perforce, the winning percentage may drop due to more teams. (Offset a little by higher win percentage from easier games with higher seeds.)

  8. #8
    Join Date
    Feb 2007
    Location
    NC
    Quote Originally Posted by mkirsh View Post
    While I mostly agree that using prior year's tournament data on current teams (especially in an era of significant player movement) would not necessarily solve the intended problem, I do think Sage's data shows that there may be systemic flaws in the current analytic framework that should be re-evaluated, modified, and back tested. So instead of just inflating the ACC, try to figure out if there is anything in the current model that is devaluing certain teams/conferences and see if those should be adjusted.
    Oh, for sure. I can totally get behind making some revisions to the system. For one thing, I think it would be useful to have a "non-con" weekend once each in January and February. This wouldn't create a perfect situation, but it would theoretically reduce somewhat the impact of the lack of interconference data after Jan 1. Which would hopefully improve the relative picture of the teams across conferences.

    But I think the reality is that there isn't ever going to be a perfect system, nor anywhere close to perfect. There are too many teams and too little time to play a full round-robin head-to-head. And even then, the dynamics of a team change over the course of a season. So there is never going to be a perfect metric. You can only do the best you can with the information you have about that season.

    And not least importantly, single-elimination tournaments are inherently fairly random. They are a LOT of fun, and they are an efficient and reasonably equitable way to determine a champion given limited resources (mainly time). But they are not great at determining who was a better team/conference. There are just too many variables and too much random fluctuation for that. So using tournament results to then back-assess how good a team was over the course of the season isn't a great tool.

    NC State is a perfect example of this. They were a flawed team that was barely over .500 at the end of the regular season, having lost at home to Duke by 15. But they caught lightning in a bottle at just the right time. They also caught a few breaks, most notably that they absolutely should have been eliminated by UVa in the quarterfinals of the ACCT. But because UVa lost its mind at the end of regulation, State was given life, and they took the opportunity and ran with it. Nobody would say that State was truly a top-4 team (nor even a top-25 team). Their tourney results didn't prove that they were underrated; it just proved that they figured something out at JUST the right time.

  9. #9
    Join Date
    Feb 2007
    Location
    Steamboat Springs, CO

    Wait! There's More!

    To give you an idea how “maniacally dysfunctional” the current system is, let’s look at the three conferences with the highest percentage receiving bids vs. the three P-6 conferences with the lowest. Over the last three years, the Big 12, the Big Ten and the SEC have had the following percent of their members receiving NCAA tournament bids: 62%, 55%, and 52% -- all over 50%. The ACC, Big East and PAC-12 have had much lower percentages: 33%, 41%, and 31% -- just about one-third..

    These must be much better conferences, right? Uh, no. The Big 12, Big Ten and SEC won only 1.4, 1.1 and 1.0 games per bid. While the ACC, Big East, and PAC-12 won 2.2, 2.1 and 1.2 games per NCAA tournament bid received.

    Ideally, the average quality of those receiving bids would be about the same across the conferences and, therefore, the number of wins per bid would be about the same. No! Those getting “stiffed” on the bids are averaging 1.82 wins per bid and those generously given bids to over one-half of their members are winning 1.15 games per bid.

    This system is not only broken, it is corrupt.
    Sage Grouse

    ---------------------------------------
    'When I got on the bus for my first road game at Duke, I saw that every player was carrying textbooks or laptops. I coached in the SEC for 25 years, and I had never seen that before, not even once.' - David Cutcliffe to Duke alumni in Washington, DC, June 2013

  10. #10
    Join Date
    Feb 2007
    Location
    Washington, DC area
    Since a big part of the problem is that there are no meaningful non-con games after December, I propose the NCAA encourage them.

    Any team that doesn't play out-of-conference after Feb 15 automatically gets moved down 10, 15, maybe 20 spots in the NET system. Ideally, it would be a game against a team in the top half of NET (but not sure how to work that in).

    -jk

  11. #11
    Quote Originally Posted by sagegrouse View Post
    To give you an idea how “maniacally dysfunctional” the current system is, let’s look at the three conferences with the highest percentage receiving bids vs. the three P-6 conferences with the lowest. Over the last three years, the Big 12, the Big Ten and the SEC have had the following percent of their members receiving NCAA tournament bids: 62%, 55%, and 52% -- all over 50%. The ACC, Big East and PAC-12 have had much lower percentages: 33%, 41%, and 31% -- just about one-third..

    These must be much better conferences, right? Uh, no. The Big 12, Big Ten and SEC won only 1.4, 1.1 and 1.0 games per bid. While the ACC, Big East, and PAC-12 won 2.2, 2.1 and 1.2 games per NCAA tournament bid received.

    Ideally, the average quality of those receiving bids would be about the same across the conferences and, therefore, the number of wins per bid would be about the same. No! Those getting “stiffed” on the bids are averaging 1.82 wins per bid and those generously given bids to over one-half of their members are winning 1.15 games per bid.

    This system is not only broken, it is corrupt.
    I think the ACC and Big East have been underrated recently but aren't both notoriously top heavy with Duke. UNC, UConn, Nova, Marquette? That would skew the wins per bid a bit. I think your analysis on wins vs projected wins based on seed was more compelling. If the ACC office doesn't have a task force working on this, they are incompetent!

  12. #12
    Join Date
    Feb 2007
    Location
    Steamboat Springs, CO
    Quote Originally Posted by SkyBrickey View Post
    I think the ACC and Big East have been underrated recently but aren't both notoriously top heavy with Duke. UNC, UConn, Nova, Marquette? That would skew the wins per bid a bit. I think your analysis on wins vs projected wins based on seed was more compelling. If the ACC office doesn't have a task force working on this, they are incompetent!
    Thanks. More work is underway on projected wins. But I hope I showed there could be a serious problem in current methods.

  13. #13
    Quote Originally Posted by sagegrouse View Post
    Thanks. More work is underway on projected wins. But I hope I showed there could be a serious problem in current methods.
    You definitely have. Enjoy the analysis. It's pretty eye opening. The folks at Wake, Clemson, Pitt, ... would probably enjoy it even more. Has impacted our seeding but has cost them bids.

  14. #14
    Join Date
    Feb 2007
    Location
    NC
    I don’t even agree with the argument that the tourney results show that the ACC was underseeded. It just shows that the tournament is fluky. I think they were properly seeded, but March Madness is, well, madness. I definitely wouldn’t call the results evidence that the selection system is corrupt.

    This year, the at large ACC teams did exactly what they were expected to do: they won 8 games and were expected to win 7. You don’t get much more accurate than that. But an enormous anomaly happened with N.C. State: a thoroughly mediocre team inexplicably caught fire in March. Not unlike the crazy Oregon St run in 2021. That doesn’t suggest to me that the ACC was underrated; it suggests that crazy stuff happens in March. Just like I don’t think the PAC-12 was underrated in 2021. Worth noting that the last at large ACC team this year got hammered in the play-in game, which doesn’t make for a compelling case that more ACC teams should get in.

    In 2022, the results were very well aligned with the expectation. Basically, the conference won 1 more game than they should have, and that 1 was a play-in game.

    In 2022, they definitely outperformed their seeds, on the backs of UNC and Miami. Those two teams won 8 games and were expected to win 1. Aside from that, the rest of the conference did as expected (with the one extra play-in game win). So let’s break down UNC and Miami. UNC was the quintessential challenge for tournament seeding: a team that played awful basketball for 3/4 of the season, then found it in February. They didn’t earn a higher seed that year, but folks knew that UNC was a dangerous 8 seed. And UNC quite nearly lost in the second round. But because March is mad, they won, and then got a 15 seed in the Elite-8. Miami almost lost in the 1st round. But they scratched out a win, and then got to play an Auburn with their star center severely injured. And then got to play an 11 seed in the Sweet-16. Those are fluky outcomes, a result of March being mad. I don’t think they don’t prove that the ACC deserved more bids in 2022 and 2024; they just show that single-elimination tournaments are chaotic. And we just happened to be the beneficiaries of the chaos in 2022 and 2024, whereas in 2021 the PAC-12 were the beneficiaries.

    I think seeding for the tournament has a ton of challenges. There aren’t enough games to exactly rate every team, and no inter conference games after December. There isn’t a great way to handle the team that catches fire late like UNC 2022. There isn’t a great way to deal with injuries. But I don’t think looking at previous years’ tourney results is an improvement over the current approach. It would just be a different flawed approach, and a more flawed approach (especially in the NIL and free transfer era in which teams will look completely different year to year). I would be all for including a few interconference game in January and February. But beyond that? I think the system is fair as is. Everyone knows the rules of the game going in. It is on the conferences to earn their bids. Play and beat good teams in the out of conference schedule and/or don’t scrape by against bad teams and the bids will take care of themselves.

  15. #15
    Join Date
    Feb 2013
    Location
    Cambridge, MA
    Quote Originally Posted by -jk View Post
    Since a big part of the problem is that there are no meaningful non-con games after December, I propose the NCAA encourage them.

    Any team that doesn't play out-of-conference after Feb 15 automatically gets moved down 10, 15, maybe 20 spots in the NET system. Ideally, it would be a game against a team in the top half of NET (but not sure how to work that in).

    -jk
    I'd go a step further. A team would need to play at least 2 non-conference games after Jan 20 (or some other date) in order to qualify for an at-large bid in the tourney. As for making sure everybody doesn't schedule 2 cupcakes, the NCAA could require at least one of the games to be Quad 3 or higher.

    Either that or use the week between the NFL Conference Championship games and the Super Bowl to have several in-season tournaments. This relative 'lull' in the sports calendar would be a great opportunity for college basketball to capture more attention before March. I've got specific ideas how this could be set up if anyone is interested.

  16. #16
    Join Date
    Feb 2007
    Location
    Athens, GA
    Quote Originally Posted by CDu View Post
    I don’t even agree with the argument that the tourney results show that the ACC was underseeded. It just shows that the tournament is fluky...
    My bias is to agree with you, CDu, and I agree that the tournament is fluky, but statistically, that just means the data is noisy. If Sage can bring the receipts with a thorough and proper analysis (not there yet I don't think), it's worth considering.

  17. #17
    While I believe tourney selection could use some fixin', I'm not sure it's possible. More analytics won't help as teams will find ways to game them, and the selection committee will find ways to apply them unevenly. A few non-con games in the later part of the season is like throwing bricks in the Grand Canyon.

    The only change I would make is teams need a winning conference record to be eligible for an at-large bid. It's a bit like shuffling deck chairs on the Titanic, but it will keep the 9th or 10th best Big-whatever team out of the tourney.

  18. #18
    Join Date
    Feb 2013
    Location
    Cambridge, MA
    Quote Originally Posted by sagegrouse View Post
    Over the past ten or more years, I have started a thread or made a post entitled, “The NCAA TSC Is on a Fool’s Errand,” referring, of course, to the lack of credible data to compare teams between conferences. The reason of course, if that almost all inter-conferences match-ups occur in the first month of the season, when teams are not at mid-season form and coaches have different approaches to bringing teams along.

    There have even been allegations that some conferences (Big 12, ahem) have intentionally scheduled and ran up the score on cupcakes to influence their overall conference rating and gain more money-yielding tournament selections.

    I would now go further and say the NCAA needs to make changes in its approach and give some credit to the past NCAA records of various conferences in its selections. And, if you say, that favors the big rich conferences, I would respond – “Well, not the SEC, whose record is 0.500 over the past three tournaments.” The Big Ten is scarcely better.

    And it is possible to take expected results for each seed into account. The good computer sci folks at the University of Illinois publish “Bracket Odds,” an updated historically based estimate of the number of wins for each seeded team, 1-16, plus the standard deviation of those wins. Thus, for each conference, based on the seed of its entries in March Madness, one can calculate wins and standard deviation of wins. (The latter is not a straightforward calculation but can be done.)

    For 2022, the five ACC teams were expected to win 5.17 games. Instead, the ACC won 14 games.

    In 2023, the five ACC teams were expected to win 5.22 games and instead won 7.

    In 2024, the five (again!) ACC teams were expected to win 7.34 games and instead won 12.

    The total for three years: expected wins – 17.7 wins. Actual wins – 33. The overall record was 33-15.

    The ACC over the past three years has greatly outperformed its NCAA seedings. The result of 33 wins would occur randomly only a few times out of 100,000 tries. (Tech. note – I converted to z-scores to more easily calculate overall std. dev.) To me, this means two things:

    (a) Our teams were seeded too low in the tournament. (I mean, ACC champ Duke as a #5 in 2023?)

    (b) Almost certainly, the ACC got cheated on the number of teams in tournament, costing big money to conference members.

    How did other conferences fare? Well, the woebegone Mountain West placed an eye-popping number of teams – six – (as Butch Cassidy exclaimed, “Who are those guys?!”). Their record this year was 4-6. The SEC was 8-8; the supposedly mighty Big 12 was 7-8. The Big East did well – with about the same number of teams as the ACC, they won 29 games. The Big Ten over the past three years has a record of 25-23. The SEC, Big Ten and Big 12 have averaged over seven selections per year. The ACC and Big East have average five.

    Surely there are questions. I can also calculate performance versus expectations for each of the other conferences, but it will take a day or two.
    While I don't think the Nov/Dec information is as useless as you seem to think, it is certainly far less than ideal. Based on results that you point out from the last 3 years, I am also coming around to the idea that the ACC may be undervalued by the committee. I am biased, but Duke seemed better than a 5 and 4 seed the past two seasons.

    On the other hand, the ACC was underwhelming in 2021 (4 total wins from 7 teams which were expected to win 6.6 total games based on seeding) and the ACC hasn't exactly excelled in the NIT in recent years.

  19. #19
    Join Date
    Oct 2009
    Location
    Durham
    Quote Originally Posted by crimsondevil View Post
    My bias is to agree with you, CDu, and I agree that the tournament is fluky, but statistically, that just means the data is noisy. If Sage can bring the receipts with a thorough and proper analysis (not there yet I don't think), it's worth considering.
    it does show a disconnect between team ability and team record. Conferences like the SEC and MWC had distributions such that mediocre teams would pick up wins over the top. You end up with teams with "enough" Q1/2 wins to get in. Contrast this with the BE, where the mediocre teams were likely stronger, but the top couple of the teams in the league were so much stronger that it was much more difficult to pick up those high-Q wins. In the end, you can't both have a seeding that is inherently win-based AND have those seedings track with how good the teams actually are. The ACC was in a roughly similar vein.

    So the conference strength is only half the story, you either need to accept that wins should not be the be-all-end-all of seeding, or give teams credit for playing top teams tough, but not winning...otherwise relatively stratified conferences like the BE and ACC will be punished for having top teams that are "too" good.

    Aside from that, the ACC needs to pull its weight out of conference. The game is the game, and the ACC hasn't been particularly good at playing it recently.
    April 1

  20. #20
    Here's what's broken to me. The committee is giving equal weighting to wins in Nov/Dec vs wins in Feb/Mar. They aren't allowing any consideration for teams improving over the season. This hurts Duke because we are so young and under Scheyer our teams have been playing our best ball late in the season. It also led to UVA being selected over Pitt this year. The best at large teams are not getting the at large bids.

Similar Threads

  1. Is this NYT April Fool’s Joke or is the NCAA a total joke
    By SoCalDukeFan in forum Elizabeth King Forum
    Replies: 13
    Last Post: 04-01-2024, 05:51 PM
  2. Bracketology: a Fool’s Errand
    By -jk in forum Elizabeth King Forum
    Replies: 2
    Last Post: 03-10-2020, 01:26 PM
  3. The NCAA Tournament Selection Committee is on a Fool's Errand
    By sagegrouse in forum Elizabeth King Forum
    Replies: 62
    Last Post: 03-20-2015, 12:38 PM
  4. The Tournament Selection Committee is on a Fool's Errand
    By sagegrouse in forum Elizabeth King Forum
    Replies: 24
    Last Post: 01-09-2012, 01:07 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •