Avery has been cited as a guy who "left too early", but it's equally plausible that he just wasn't an NBA talent, and that he left at exactly the right time i.e. when he was in a position to parlay a terrific sophomore season into a lottery pick.
Except he wasn't in the lottery; he was drafted just out of it at #14 by the Timberwolves. He could have gone much lower if NBA GMs weren't infatuated with Duke players at the time. That draft was heavy on PGs with Francis, Davis, Miller, and Terry all ahead of Avery on the draft board. As the Timberwolves also had Terrell Brandon and Chauncey Billups, Avery didn't get off the bench much and never made it to a 2nd contract. If he had waited a year, the 2000 draft was much weaker at PG. Kenyon Dooling at #10, Mateen Cleaves at #14, Speedy Claxton at #20, Erick Barkley at #28 were the only ones to go in the first round. That draft overall was weak. I think he could have been a mid-lottery pick and ended up with a team much more dedicated to developing him.
Then again, Avery came from a poor background and that certainly factored into his decision to go pro.