I feel similarly as to how I feel about pro sports Halls of Fame. It's for the GREAT, not the very good. (or it should be IMO) We can't, and shouldn't, honor FOREVER and permanently in this most elevated way, players who weren't truly elite for their time, and that's why the criteria of either being an All-American or a significant Duke record holder seem to make sense. Nolan Smith and Mark Alarie and Jim Spanarkel were all really, really good players and important pieces of Duke history. But they're not quite Hall of Famers, not at a place with the history that Duke has.
And for sure, things like NBA success, being a good announcer, or a good coach should have nothing to do with these decisions. This is an individual recognition of elite and sustained and exceptional performance on the basketball court while at Duke.