Quinn Cook start?

[CDu]

So Pomeroy's entire point in that article is sample size? It's oddly written, if that's the case. But if so, here's my next question: he says he ran his example based on 70 possessions a game. And seems to have determined the plus/minus on a per possession basis. So why are you saying the player only has 30 data points? Why isn't there a data point for however many possessions the player is on the floor? Seems to me for a team we're talking about ~2000 data points per season and for a player who plays 30 minutes a game we're talking about 1500. And yes, 2000 is bigger than 1500, but not so much bigger that one is totally reliable and one completely unreliable. What am I missing?

After re-reading what he wrote (it had been a while since I read it) I do think sample size is part of the issue. Though I don't think it has much to do with sample size as I originally suggested. I'm not sure why he doesn't report the cumulative results of the 50 simulations (to create a 1000 game, or 35000-possession, simulation) rather than each 20 game simulation. Over those 1000 games, the player's +/- should theoretically have approached 0.

But I think his analysis still clearly illustrates the potential lack of value of individual +/- in single games (or even 20-game seasons). An irrelevant player (and I think we can agree that the hypothetical player in Pomeroy's example was irrelevant) can quite clearly have some odd results.

I think the other issue is the difference between team measurement and individual measurement. There are just so many more factors outside of the individual's control that can influence his +/- than there are for a team. So the sample size issue is more meaningful, because you need more data points to adequately control for the additional variables. If you had the exact same number of data points for a player and a team (which in almost every case we don't), the +/- for a team would still be a more accurate estimate than the +/- for an individual, because there are more variables to account for with the individual.

Estimating adjusted +/- would be an attempt to control for some of those additional variables (control for the lineup quality of his team and the lineup quality of opponent). But we don't have those data.

[Kedsy]

First off, to understand what Pomeroy did, you need to understand how he's analyzing +/-. Knowing a team is +5 with a player on the floor doesn't tell you anything....perhaps the team would be +5 with those 4 players playing shorthanded! To truly analyze how valuable a player is, we in statistics would prefer IF POSSIBLE (Key words here) to use a method known as WOWY - With or Without You. In a perfect sense, you'd compare the player on the court with four teammates to the performance of the team with the Player OFF THE COURT with the same four teammates still playing.

After re-reading what he wrote (it had been a while since I read it) I do think sample size is part of the issue. Though I don't think it has much to do with sample size as I originally suggested. I'm not sure why he doesn't report the cumulative results of the 50 simulations (to create a 1000 game, or 35000-possession, simulation) rather than each 20 game simulation. Over those 1000 games, the player's +/- should theoretically have approached 0.

But I think his analysis still clearly illustrates the potential lack of value of individual +/- in single games (or even 20-game seasons). An irrelevant player (and I think we can agree that the hypothetical player in Pomeroy's example was irrelevant) can quite clearly have some odd results.

I think the other issue is the difference between team measurement and individual measurement. There are just so many more factors outside of the individual's control that can influence his +/- than there are for a team. So the sample size issue is more meaningful, because you need more data points to adequately control for the additional variables. If you had the exact same number of data points for a player and a team (which in almost every case we don't), the +/- for a team would still be a more accurate estimate than the +/- for an individual, because there are more variables to account for with the individual.

Estimating adjusted +/- would be an attempt to control for some of those additional variables (control for the lineup quality of his team and the lineup quality of opponent). But we don't have those data.

Thank you both. I'm pretty sure I already more or less understood how Pomeroy is analyzing it. And the factors you both bring up sound completely reasonable. The disconnect for me is I don't see Pomeroy relying on the factors you are noting. It seems to me he shows a high amount of random noise, which would seem to apply to both individual plus/minus and his team metrics, and then dismisses plus/minus as a meaningful stat because of the noise. Perhaps in the back of his mind he was thinking all the things you're saying, but he doesn't mention them. Which means his article is either poorly conceived or poorly written, I don't know which. Usually his analysis is so complete and so cool but this one doesn't seem up to his usual standard.

Why does it matter whether this article is any good? Because every time the subject of plus/minus comes up on DBR, someone cites this Pomeroy article and says that's the end of the discussion. And I don't think it should be. Anyway, thanks for indulging me.

[yancem]

Ok, so I just read through the the past two pages and what I got out of this is, will Jumbo be joining us the season. You guys can argue the stats all you want, (I have to rely on my eyes because I promised myself to leave math in college )

[CDu]

Thank you both. I'm pretty sure I already more or less understood how Pomeroy is analyzing it. And the factors you both bring up sound completely reasonable. The disconnect for me is I don't see Pomeroy relying on the factors you are noting. It seems to me he shows a high amount of random noise, which would seem to apply to both individual plus/minus and his team metrics, and then dismisses plus/minus as a meaningful stat because of the noise. Perhaps in the back of his mind he was thinking all the things you're saying, but he doesn't mention them. Which means his article is either poorly conceived or poorly written, I don't know which. Usually his analysis is so complete and so cool but this one doesn't seem up to his usual standard.

I disagree a bit here. The point of his article wasn't to illustrate why his methodology is superior. It was merely to show that +/-, while a fun new toy, isn't a very good tool (especially single-game +/-). In that regard, I think it did a very good job.

Why does it matter whether this article is any good? Because every time the subject of plus/minus comes up on DBR, someone cites this Pomeroy article and says that's the end of the discussion. And I don't think it should be. Anyway, thanks for indulging me.

I don't the lack of discussion of his own methodology invalidates his article as evidence that single game +/- are meaningless. Remember - the whole point of the article is to show that single game +/- is meaningless (and possibly full-season +/-). His analysis shows this pretty well.

Your question about his math is a perfectly logical next question, and his article clearly doesn't address that. But it doesn't invalidate the use of the article as evidence that single game +/- are meaningless.

[-jk]

My biggest issue with +/- goes one step beyond random chance to the individual matchups.

Not every teammate is the same, nor is every opponent the same, both on a team and between teams, sometimes to a player's advantage. Sometimes not.

Zone v. man. A mobile big vs. one that parks in the middle. A pass-first v. shoot-first PG. The possibilities are almost endless, and the average player won't be equal to all opponents.

In the nba, where the skills are more consistently high and you play teams more then once, you may be able to work some of the variability out. In college, much less so.

-jk

[Wander]

Not every teammate is the same, nor is every opponent the same, both on a team and between teams, sometimes to a player's advantage. Sometimes not.

Agreed. Because of this, I think plus/minus stats are only useful for comparing guys who play the same position. And even then, there are complications.

As for the referenced article, I absolutely love the kenpom-type stuff, but in my opinion he and others put a bit too much faith into the idea that basketball players and teams are weighted random number generators.

[Kedsy]

I don't [think] the lack of discussion of his own methodology invalidates his article as evidence that single game +/- are meaningless. Remember - the whole point of the article is to show that single game +/- is meaningless (and possibly full-season +/-). His analysis shows this pretty well.

Your question about his math is a perfectly logical next question, and his article clearly doesn't address that. But it doesn't invalidate the use of the article as evidence that single game +/- are meaningless.

Well, even without the article I think common sense would suggest single game +/- are meaningless. The feeling I got from the article is he is going further to suggest all +/- analysis is meaningless (at least in college where the season is shorter and you only play each opponent once or twice). That's certainly the way a lot of DBR posters seem to view (and use) the article.

I guess my problem with this is a team's tempo-free performance in a single game is almost as useless as a player's +/- rating. I accept your and Loran's reasoning for why a team performance might be more reliable than a +/- rating, but there are still a great many random factors in a single-game result. My point is if we're going to dwell on the uselessness of single game +/- stats, why look at any single game stats at all? It's the reliance on some but not others that rankles me.

[OldPhiKap]

My biggest issue with +/- goes one step beyond random chance to the individual matchups.

Not every teammate is the same, nor is every opponent the same, both on a team and between teams, sometimes to a player's advantage. Sometimes not.

Zone v. man. A mobile big vs. one that parks in the middle. A pass-first v. shoot-first PG. The possibilities are almost endless, and the average player won't be equal to all opponents.

In the nba, where the skills are more consistently high and you play teams more then once, you may be able to work some of the variability out. In college, much less so.

-jk

QUIT USING LOGIC!!!

Stats > Coach observations. It's proven. Just Google it.

[-jk]

Well, even without the article I think common sense would suggest single game +/- are meaningless. The feeling I got from the article is he is going further to suggest all +/- analysis is meaningless (at least in college where the season is shorter and you only play each opponent once or twice). That's certainly the way a lot of DBR posters seem to view (and use) the article.

I guess my problem with this is a team's tempo-free performance in a single game is almost as useless as a player's +/- rating. I accept your and Loran's reasoning for why a team performance might be more reliable than a +/- rating, but there are still a great many random factors in a single-game result. My point is if we're going to dwell on the uselessness of single game +/- stats, why look at any single game stats at all? It's the reliance on some but not others that rankles me.

I have a similar problem with the team stats. Different teams and injuries can wreak havoc on predictors.

These numbers, at least, can be tested by comparing the predicted outcomes with real outcomes. I'm sure it's done to tweak systems, but I'm not about to hunt them down.

I can think of no meaningful way to test +/- for any sort of validity. Much as with a box score, there are measureable numbers, even if they can't paint a full picture. The question is how meaningful the picture can be. Perhaps the mathmetical luminaries can find a way to enlighten the rest of us.

-jk

[CDu]

Well, even without the article I think common sense would suggest single game +/- are meaningless.

I'd agree, except that people on DBR regularly use single-game +/- to say "see, player X actually played well." And I doubt it's exclusive to DBR. So clearly the point needed to be made at some level.

The feeling I got from the article is he is going further to suggest all +/- analysis is meaningless (at least in college where the season is shorter and you only play each opponent once or twice). That's certainly the way a lot of DBR posters seem to view (and use) the article.

I don't think that's the case, though it may be for some. I've only ever referenced it in terms of single-game +/-. I'm skeptical of even full-season, but I'm more accepting of full-season. And yes, Pomeroy is also skeptical of full-season +/-, which is a part of the article (point #2, I guess). He's less absolute about this.

I guess my problem with this is a team's tempo-free performance in a single game is almost as useless as a player's +/- rating.

I agree. I think Pomeroy agrees too. I believe he readily admits that the first couple of months of his data are highly variable and that it doesn't settle in as a tool of value until the second half of the season. And that's despite the fact that his approach makes much more of an effort to adjust for outside influences than +/- does.

I accept your and Loran's reasoning for why a team performance might be more reliable than a +/- rating, but there are still a great many random factors in a single-game result. My point is if we're going to dwell on the uselessness of single game +/- stats, why look at any single game stats at all? It's the reliance on some but not others that rankles me.

There's a difference here. Looking at stats like points, rebounds, etc, are all tangible measures of an individual's performance. They aren't perfect (a player can luck into stats), but they're at least tangible measures. You know that player X scored X points. What people are trying to do with single-game +/- is show the intangible impact a player had. But you can't separate the intangible impact (or lack thereof) from randomness or other factors not related to the player. So it makes more sense to stick with the tangible stats than the +/- when looking at a single game.

[superdave]

A writer over at Scout makes a strong case based on Bellarmine. Let's see what happens vs. Shaw.

[Kedsy]

A writer over at Scout makes a strong case based on Bellarmine. Let's see what happens vs. Shaw.

Well, how strong his case is probably depends on how you feel about the plus/minus debate we've been having for the past several pages of this thread.

[superdave]

Well, how strong his case is probably depends on how you feel about the plus/minus debate we've been having for the past several pages of this thread.

I dont want to wade into that debate a whole lot, because +- is not the all-stat or anything. But I think it's at a minimum useful. It also backs up Coach K's quotes after the Bellarmine game.

If Thornton continues to play well, it leads to two questions-

Will Thornton's play be as good as his usage increases?

Can Thornton score enough to justify taking minutes from (seemingly) better offensive players? TT shot .448 on FGs, .267 on 3's and had an assist to TO ratio of 33:22 last year.

[Kedsy]

I dont want to wade into that debate a whole lot, because +- is not the all-stat or anything. But I think it's at a minimum useful. It also backs up Coach K's quotes after the Bellarmine game.

If Thornton continues to play well, it leads to two questions-

Will Thornton's play be as good as his usage increases?

Can Thornton score enough to justify taking minutes from (seemingly) better offensive players? TT shot .448 on FGs, .267 on 3's and had an assist to TO ratio of 33:22 last year.

As I mentioned on the Phase thread, I'm a little concerned that if Tyler plays a lot our overall three-point shooting will suffer. I'm not convinced he will get the ball to Austin and Seth when and where they need it to be most efficient (and of course if Tyler's out there with Austin and Seth, then Andre is on the bench). Or, put another way, while the tradeoff with Tyler is getting defense in exchange for offense, but it's not his individual offense I'm worried about as much as the possibility of the team's offense bogging down with him at the point. We haven't really seen much of him running the team, however, and I assume he's improved from last year, so it may be premature at this point to judge his ability to run the team.

[CDu]

I dont want to wade into that debate a whole lot, because +- is not the all-stat or anything. But I think it's at a minimum useful.

I disagree here. I think it's not useful, for all the reasons that have been discussed. There are certainly going to be times when it's consistent with the "eye test" (or more importantly Coach K's observations). But that doesn't make it useful.

In this case, it may very well be that the team had better +/- as a result of Thornton being on the floor. It may also have been a coincidence driven by other factors (or random luck). In other words, saying that single game +/- is useless doesn't mean that Thornton didn't actually play better than the alternatives - it just means that single-game +/- isn't the best argument of this.

If Thornton continues to play well, it leads to two questions-

Will Thornton's play be as good as his usage increases?

Can Thornton score enough to justify taking minutes from (seemingly) better offensive players? TT shot .448 on FGs, .267 on 3's and had an assist to TO ratio of 33:22 last year.

I think these are all reasonable questions. Who handles the ball? Thornton isn't a great shooter, and he's also not a great playmaker. Meanwhile, Rivers and Curry seem very capable of creating their own shot. Theoretically, a guy like Dawkins (a catch-and-shoot threat) makes more sense, because defenses can't cheat off of him to help on Rivers/Curry. Alternatively, I imagine good teams would play off Thornton and make him beat them (or make Rivers/Curry work that much harder to beat them if they don't pass to Thornton).

Also, does our defense suffer against better competition (especially against a bigger team) with a smaller perimeter group?

These questions aren't to say that Thornton isn't worth a look. It may be that Thornton's superior floor savvy offsets the size difference between him and Dawkins/Gbinije/Murphy. And it may be that the offensive issues aren't a concern either. I just wonder about them in such a lineup.

[CDu]

As I mentioned on the Phase thread, I'm a little concerned that if Tyler plays a lot our overall three-point shooting will suffer. I'm not convinced he will get the ball to Austin and Seth when and where they need it to be most efficient (and of course if Tyler's out there with Austin and Seth, then Andre is on the bench). Or, put another way, while the tradeoff with Tyler is getting defense in exchange for offense, but it's not his individual offense I'm worried about as much as the possibility of the team's offense bogging down with him at the point. We haven't really seen much of him running the team, however, and I assume he's improved from last year, so it may be premature at this point to judge his ability to run the team.

I'd be surprised if Thornton ran the offense in a pairing of Curry/Rivers/Thornton. I'd be more inclined to believe that Curry and Rivers would still initiate the offense for themselves and others. Thornton might take some of the load off in bringing the ball up the court, but when we get to the half-court I'd expect him to move to the wing. So I'd say the question about offense still applies, just in a slightly different way.

Granted, I'm also basing this off what I saw last year and in minimal time early this year, so I apply the same caveat as you to my logic. It could be that he would indeed run the offense (in which case I'd have the same questions as you), or it could be that he's dramatically improved (in which case neither concern ends up materializing).

[superdave]

I'd be surprised if Thornton ran the offense in a pairing of Curry/Rivers/Thornton. I'd be more inclined to believe that Curry and Rivers would still initiate the offense for themselves and others. Thornton might take some of the load off in bringing the ball up the court, but when we get to the half-court I'd expect him to move to the wing. So I'd say the question about offense still applies, just in a slightly different way.

Granted, I'm also basing this off what I saw last year and in minimal time early this year, so I apply the same caveat as you to my logic. It could be that he would indeed run the offense (in which case I'd have the same questions as you), or it could be that he's dramatically improved (in which case neither concern ends up materializing).

I think the best use of Tyler will be to reverse a bad defensive trend, to change the pace of the game by speeding up the other team via his defensive energy, or to lock down a specific player from the other team. I think those uses are great for what he excels at, and they are likely to be used in spurts.

Also, Tyler's energy and attention to defense can serve as a model for the other teammates. Play D like Tyler and your playing time will increase.

Finally, the offensive concerns with Tyler running the point are pretty big. I agree that he'd likely let one of the other guards initiate the offense. I think the difference for him between spot duty and 15 minutes a game is knocking down his jumpers consistently. You know the D will be full of effort (and hopefully not full ofreach in fouls), but you dont really know what the O will look like.

[sagegrouse]

As I mentioned on the Phase thread, I'm a little concerned that if Tyler plays a lot our overall three-point shooting will suffer. I'm not convinced he will get the ball to Austin and Seth when and where they need it to be most efficient (and of course if Tyler's out there with Austin and Seth, then Andre is on the bench). Or, put another way, while the tradeoff with Tyler is getting defense in exchange for offense, but it's not his individual offense I'm worried about as much as the possibility of the team's offense bogging down with him at the point. We haven't really seen much of him running the team, however, and I assume he's improved from last year, so it may be premature at this point to judge his ability to run the team.

Having studied under Machiavelli in my youth, when K makes comments like he did about Tyler, I hear a subtext. K said, "Thornton really gave us a big boost. When he's in the ball game, we just play better. He doesn't have to hit a shot - we just play better when Tyler's in the basketball game."

Does anyone else hear the following?

1. "Tyler is the feistiest player on the team and never backs down from anyone. I like feisty, and if he's the only one who plays that way, he's gonna get a lot of minutes."

2. "Dammit, I said 'defense' is important! That's what we get when Tyler's on the court. Do you hear that, three point shooters? We are going to play defense, even if the final score is 10-8!"

3. "Uh, Tyler -- Tyler? I'm covering for you because you play hard and help the team. But if you aren't going to 'hit a shot,' you are going to have to pass the ball regularly to players who are. And, by the way, I think you can hit a few shots. You aren't afraid to shoot, are you?"

sagegrouse

[Kedsy]

I think the best use of Tyler will be to reverse a bad defensive trend, to change the pace of the game by speeding up the other team via his defensive energy, or to lock down a specific player from the other team. I think those uses are great for what he excels at, and they are likely to be used in spurts.

I agree with your first two points but not necessarily with the bolded one. I think CDu's right when he has said that Tyler isn't a great on-ball defender (which is what I expect when I hear "lock down" defender). His defensive strengths lie in things like getting into the passing lanes to disrupt the other team's offense, hustling to make an opposing player work really hard to get the ball, taking a charge (usually from someone other than his own man). All these things are done off the ball, and he's great at that. But I wouldn't necessarily call on him to lock down someone who's been lighting us up. I'm not sure he's quick enough for outstanding on-ball, one-on-one D. Obviously he could prove me wrong, but I don't think we've seen it thus far.

Having studied under Machiavelli in my youth, when K makes comments like he did about Tyler, I hear a subtext. K said, "Thornton really gave us a big boost. When he's in the ball game, we just play better. He doesn't have to hit a shot - we just play better when Tyler's in the basketball game."

Does anyone else hear the following?

1. "Tyler is the feistiest player on the team and never backs down from anyone. I like feisty, and if he's the only one who plays that way, he's gonna get a lot of minutes."

2. "Dammit, I said 'defense' is important! That's what we get when Tyler's on the court. Do you hear that, three point shooters? We are going to play defense, even if the final score is 10-8!"

3. "Uh, Tyler -- Tyler? I'm covering for you because you play hard and help the team. But if you aren't going to 'hit a shot,' you are going to have to pass the ball regularly to players who are. And, by the way, I think you can hit a few shots. You aren't afraid to shoot, are you?"

sagegrouse

Yes.

[CDu]

I think the best use of Tyler will be to reverse a bad defensive trend, to change the pace of the game by speeding up the other team via his defensive energy, or to lock down a specific player from the other team. I think those uses are great for what he excels at, and they are likely to be used in spurts.

I agree with the rest of what you said, but I don't think locking down a specific player is one of Thornton's strengths. I don't think he's all that great at one-on-one defense. I think he's a terrific defender in other areas (reading passing lanes, helping on switches, general court awareness). And he may still be one of our best perimeter on-ball defenders (by default). But I don't think "lock down defender" is the right choice of words.

Edit: it appears I should have read another couple of posts down, because I basically just repeated what Kedsy said.