I typically agree with this sentiment. You're deciding the game by not calling the rules, but, in that case, I would not have called that foul. I'm not even saying that's absolutely the right way to be about it. I'm just saying that, in that situation, the odds of a ref calling that foul are slim to none. And I think even that might be too optimistic.
Let's just thank whatever we consider holy that the shot didn't go in. Dang thing was so close, every time I see it I still think it's going through the hoop.