As Dave Berri helpfully reminds us, the 7-game NBA playoffs do not constitute an experiment of statistical significance*.
I obviously agree with this observation, and it gladdens me to see Berri illuminate this aspect of the NBA playoffs. To quote the title of a post from May, "The Playoffs are for fun, not science". Well put, Devin.
As much as we'd love to pretend otherwise, Game 6 of last year's NBA Finals showed us that the line between a champion and a runner-up is perhaps as thin as a single carom gone awry... and into the perfect place for a ridiculously clutch corner 3. An entire season depends on the bounce of a ball. How eerily fitting for Berri's argument. After all, that's an extreme and explicit example of the natural indeterminacy of sports: There are plenty of chance events that can completely determine the outcome of a season: Whether a player lands on his side or on his arm can change the next 2 years of that player's career and, if he's sufficiently important, his team's success. Whether a player hits the ball at one angle or another may determine whether his hand is broken. Whether a lottery ball hits the right slip-stream can determine a generation of games.
Obviously in all of these examples, the dominant factor is not the executive, the coach, or potentially even the players involved. Fluky events that change whole eras of basketball, man. It's crazy. Prof. Berri really got me thinking.
In fact, I can't help but notice this paragraph from an article on tanking this same Prof. Berri also penned.
The tanking strategy is easy for decision-makers in the NBA to embrace. Teams that pursue this strategy are essentially trying to lose to enhance the team’s draft position. This is a simple strategy to follow. Trying to win is difficult, but losing is easy and the more incompetent the decision-maker, the better the strategy can be implemented. Imagine how easy it would be to do your job if you were rewarded for doing the job badly!
To stop this behavior, the NBA could simply implement a rule that says if a team misses the playoffs for three consecutive seasons, the team must fire its general manager. If this rule was put in place, constant losing would lead to consequences for executives.Obviously Berri isn't talking about getting rid of bad executives here. After all, with all the variation in team quality being so vast, and the difficulty of rebuilding being nigh impossible without some breaks going your way? How could 3 seasons ever accurately constitute a statistical experiment designed to weed out below-average executives? The answer is obvious: They couldn't! This rule takes three Bernoulli trials - each possibly as susceptible to chance as a single coin flip or two - and makes it into a GM's livelihood.
So, in the absence of the statistical significance that is enough to dismiss an entire two-month stretch of the season, Prof. Berri's suggestion should obviously then be taken as an amusing, populistic guideline rather than something to be literally enacted. Indeed, his thoughts are more like signals that might indicate good ideas but that need the rigor of a more analytic mind to prove or even formulate correctly.
I find that the addition of this perspective allows me to better appreciate his contribution to analytics. Instead of the arrogance and intellectual laziness I'm inclined to ascribe to his writings, stepping back has allowed me to temper my anger and recognize that he is not writing as an academic but as a thinker. He writes think pieces, not math pieces. He is trying to show us how fun thinking can be and how cool some things in academia are instead of actually understanding and conveying them correctly. Basketball is a complicated game and the analytics are getting more interesting every day, but when I don't have time or sophistication to catch up on the actual advances I can go to his writings for a watered-down version of these things, written with the same comforting confidence of the people on the actual frontiers of understanding, among which are several of his blog's co-authors.
And, just so you aren't inclined to completely dismiss my uneducated ramblings, you should note that after having read hundreds of his articles, my own perspective on this matter is quickly approaching statistical significance, which, as you know, is the objective truth, absolute and unadulterated.
*Statistical significance is defined as that totally-not-arbitrary <5% p-value that you get to call "statistical significance" without qualification and have it technically be true on face value
Yea I prefer the other WP posters like Andres and Arturo to Berri, mainly due to Berri speaking/writing with an arrogance and almost anger that’s unnecessary. Berri just seems to miss things sometimes, aside from the clearcut fixes to WP that’s never been implimented - for example saying academic essays get critiqued by peers and blog posts don’t, so blog posts aren’t as valid writing/ideas as them. Without seeing the obvious connection that the relationships/agendas/pressure to conform/BS-ing inside academic circles, makes critiquing each other’s work about the least trustworthy way to judge something. Or for example thinking awarding Best Picture to the highest grossing box-office film would mean the public decides what the best film is, without seeing how much of box-office success is related to other variables than people liking the film. When I see stuff like that it makes me distrust his thought process, even if it’s unfair to single out a few examples.
ReplyDeleteI think Berri has been valuable to the bball community for his brashness and for getting a question the status quo mentality out there - and WP is at least tilted towards the right direction in ppg is bad, efficiency is good, even if they get screwed up after that with the rebounding, defensive adjustment, etc.
Yeah, that's about right. I honestly have been thinking about the fixes I'd make to WP (was it ASFW that had the "replacement shooting efficiency"?). But more than that, it's a decent metric and a decent group of people, and I haven't lost sight of that. The apparent arrogance in dberri's own writing is pretty frustrating and is basically my biggest frustration with WoW. This piece is intended as satire but I think there's a lot of truth to my presentation. I simply don't see the rigor on a day-to-day basis, no matter what his academic chops. And arrogance without rigor is the worst of both worlds.
DeleteYou know, I have to think that "rigor" is the new frontier and will soon be the new baseline, and people that are willing to be open and honest about the limitations of their systems are going to be the ones with the most ability to make stronger claims. We're going to have all this data available to us and we can combine it in far more interesting ways than ever before, and we're largely past the point where people doubt whether analytics has anything to offer. But if it doesn't come with rigor (either in public presentations like blogs or behind closed doors), then the gains are bound to be misleading. People always joke about "small sample size" this time of year but it's not clear that we really have a good grasp for what a solid sample size actually is when we make those claims!
I think WP benefits from being simple in that it is probably measuring *something* and the other posters certainly to a great job explicating and helping us visualize what WP measures. More sophisticated systems like RAPM benefit from, well, their sophistication, but it's unclear at times if they're actually meaningful. And they tend to be more obscure and baroque in their presentation. So maybe the answer is to open the lines of dialogue, add error bars and sweet-ass visualization to everything, and, in the end, to "hustle and grind before the other be findin us."