I like Malcolm Gladwell. Malcolm Gladwell likes sports. My friend Harsh doesn’t like Malcolm Gladwell.
No, that’s not a test of the transitive property, although Harsh doesn’t like spectator sports. Harsh says Gladwell is a pseudo-scientist, that his work is sloppy and his conclusions often flawed. So when Jeremy Britton defended a preseason prediction by Wins Produced (WP) that Golden State would win 57 games this year, my thoughts turned to Harsh and Gladwell.
In 2006, Gladwell wrote a New Yorker article and this subsequent blog post advocating Wins Produced. As I mentioned in my overview of popular advanced basketball statistics, Wins Produced has serious problems as an individual player valuator. Yet despite its massive shortcomings, Gladwell endorsed it. He wrote:
Here’s what I think the real value of the Wages of Wins system is, though. It gives us a tool to see those instances where our intuitive ratings of players may be particularly inaccurate. In my New Yorker piece, I focused on how the algorithm tells us that Allen Iverson isn’t nearly the player we think he is.
That would be interesting and true…if Wins Produced were a good tool. It’s not. It doesn’t actually tell us when our intuitive ratings of players are off. If it did, it would be valuable as a predictive tool. Only, when big elite WP performers change teams, nothing much out of the ordinary happens.
Sometimes, someone like Michael Smith leaves his team and nothing happens at all. At 28, Smith led all full-time players in WP per 48 minutes in 2001, and never made another NBA team again. It wasn’t a case of WP telling us something new and informative, it was just another case of WP generating yet another ridiculous result.
Ah, but how does it produce such outliers? (The full calculation is explained here in detail.) In my previous assessment of the metric, I provided an example of how problematic the marginal values assigned by Wins Produced can be. (eg, how much value it assigns to a rebound or a missed shot.) Those values are generated by a regression on NBA team data, in an attempt to figure out what statistics correlate with winning. It then distributes credit based on those correlations, and does a bit of hand-waving to fit the final result as closely as possible to team wins.
But there are problems with the results because there are problems with the method. Namely:
- It’s correlative
- It’s limited by the box score
As anyone who has taken College Class 101 knows, correlation does not equal causation. It turns out, team attendance is highly correlated with winning percentage:
Who knew that all an NBA team had to do in order to win was fill the seats!
Then there’s the problem that the NBA box score doesn’t explain everything in the game, mostly ignoring defense. So Dave Berri — one of the founders of the model — crunched some numbers and determined defensive rebounding is related to winning. Only he, nor any WP proponents, apparently, stopped to think about why.
Good defensive teams win basketball games. Good defensive teams force misses. As we know, after a miss there is a rebound. The miss causes the rebound. The rebound doesn’t cause the miss. Unless you think the large crowd causes a team to win, that is.
The correlation coefficient between eFG% against and team defensive rating last year was a whopping 0.89…only there is no traditional individual metric for missed shots against. So the fruits of that labor show up in the rebounding numbers.
I wonder, though, if WP proponents think opponents miss shots because players are grabbing rebounds.
What’s surprising about Gladwell’s endorsement of Wins Produced isn’t that it takes a basketball expert to either (a) notice the problems with the results or (b) notice the problems with the method, it’s that anyone taking a small amount of time to read about the statistical model can see its gaping holes.
Gladwell, for my money, is a fantastic story teller. He’s a macroscopic thinker and his ideas are thought-provoking, even when raw or unsubstantiated. But it seems he gets into trouble when he probes too deeply into esoteric fields. After all, he can’t be an expert in everything, so how can he judge the quality of his expert sources?
If Gladwell dug deeper, he would have found that Wins Produced isn’t exactly popular in basketball analytics circles like APBR. He would have found that Berri blocks dissenting opinion from his blog — I was blacklisted in one day after this bizarre exchange with him. When the 2001 NBA MVP is the ninth best player on his own team, either all the coaches and writers are missing something, or Wins Produced is.
Sadly, Malcolm chose the former explanation.
(Note: A toy metric derived in the same manner as WP and able to make the same claims was made in about 10 minutes by this basketball blogger. It’s worth a good laugh.)