Michael Lewis’ 2004 bestseller, Moneyball, subsequently made into a movie in 2011, has entered the vernacular. We have Moneyball for government, Moneyball for writing (and awarding) grant proposals, and, yes, Moneyball for lawyers. And its basic premise is hard to argue with: Data–especially “Big Data,” where you can find it–if properly analyzed, doesn’t lie. Another factor has, I suspect, heightened its impact on business, its visibility in popular culture, and its entry into the common parlance of conversation: Lewis published the book right around the same time behavioral economics, with all its learning about systematic biases human judgment and intuition, also came to the fore (loss aversion, anchoring, confirmation bias, heuristics, framing, and all the familiar rest of it).
The combination is killer: Data can tell stories you never knew were there to be discovered, and your own gut can lie to you about reality.
So what more is there to be said?
Lots, it turns out.
Our texts for today are a two-part interview series in the McKinsey Quarterly featuring Houston Astros general manager Jeff Luhnow after his team won the 2017 World Series. (I. How the Houston Astros are winning through advanced analytics, June 2018, and II. A view from the front lines of baseball’s data-analytics revolution, July 2018) Four years earlier they had lost whopping 111 games in the 162 game season (68%).
Nor did they do it through check-writing prowess: Their payroll was 18th of the 30 major league teams, and almost 50% ($118-million) less than the highest spenders, the LA Dodgers.
Putting it mildly, this turnaround was no accident.
Now, you may think you know everything you need to about the Moneyball approach–and have concluded it could never apply to your firm because (a) lawyer performance isn’t reducible to data; (b) even if it were we haven’t collected the data; (c) if we wanted to collect “the data” we’d have no idea exactly what data we should be looking for or tracking; (d) even if we knew what to get, had it, and could evaluate lawyers using it, they’d never accept it because human judgment is, y’know, ineffable; and (e) you get the idea.
When Luhnow arrived at the Astros in 2011 the situation was very similar.
Quarterly: What were the analytics strengths and weaknesses for the Astros when you joined them in 2011?
Luhnow: There really was not any focus on analytics at all. It was a traditional scouting organization. … [I]n terms of the analytic capabilities of the organization, if I were to rank it, Houston would have been in the bottom five for sure.
Quarterly: Were the existing personnel receptive to your changes?
Luhnow: No. There are hundreds of people that work in a baseball organization, including coaches, scouts, and hundreds of players that are signed at any one point in time. They did not accept it (emphasis supplied)
So how did Luhnow push this cultural boulder uphill?
That’s where the story gets interesting and, I hope, instructive.
Major League Baseball isn’t Big Law, but you can feel viscerally how the organization took to the changes analytics introduced.
Quick digression on one of the techniques of analytics in MLB: Because you have data on where any given batter has traditionally been able to hit the ball effectively [down the third base line, through the gap between first and second base, etc.], you can “shift” your fielders defensively to be in position to anticipate that batter’s favorite probability dispersion of hits. This obviously moves them away from the traditional position where, say, the shortstop “ought to be.”
Here’s a real heatmap of a hitter’s favorite target zones:
Now the following should make more sense.
I’ll give you a great example. The pitcher’s on the mound; he throws a pitch. The ball gets hit to where, for the pitcher’s entire career, there’s been a shortstop right behind him. But all of a sudden, the shortstop’s not there, because the analytics would tell us the shortstop should be on the other side of the base. So, to that pitcher, that’s a massive failure—that ball should’ve been an out, and instead, that ball turned into a base hit and maybe a run that’s going to go on his personal record.
People always remember the negatives. It’s harder for a pitcher to remember the ball that got hit up the middle that, in years past, would’ve been a single, but this year, it just so happened the second baseman was right there, stepped on the base, and got a double play. We get a little less credit for those, though, than we get dinged on the negative ones.
It’s hard to convince the pitchers that this was the right thing to do. Because it was so different. It felt wrong. The defense wasn’t standing in the positions that they’ve been standing in since these guys were in Little League. Pitchers would therefore glare into the dugout and glare at the coaches that asked infielders to move, or glare at the infielders themselves. And over time, everybody would go back to their traditional positions. That was the first year.
The second year the coaches “were a little bit more forceful” about having the players shift, and they “did a nice job for the first couple of months,” but infielders and pitchers started complaining that they weren’t used to it and by the end of the season Houston was back in the middle of the pack in terms of the shift.
It took a change of heart by one of the team leaders to alter the dynamic. This happened during spring training before the start of the third season on “analytics” when the managers shared the theory and empirical evidence behind analytics with the pitchers and fielders:
There was an incredible moment where one of our younger pitchers who really wasn’t quite getting it kept complaining, “Well, what about this? What about that?” One of the veteran pitchers who had come around turned to the younger pitcher and said, “Look, this is going to help you have a better ERA [earned run average] and have a better chance to have a better career, so you should really take this seriously.” Once you start getting players to advocate for the use of these tools, it changes the whole equation. Because then you’re no longer pushing; it’s starting to pull. Once that happens, the sky’s the limit in terms of the impact that these technologies and analytics can have on the players. (emphasis supplied)
So now that with analytics the Astros have won the World Series, are they done? Of course not, but we’ve actually heard law firm partners say that they’ve already changed so they’re done. (I kid you not.)
The general question is, how do you stay in the lead?
It involves being far enough out front of your peers that you inevitably make mistakes.
We talk internally about being on the “bleeding edge.” We know we’re going to have some cuts, some nicks, some bruises—because if we’re not, it’s similar to base running. If you have a player on first, and he never gets thrown out at third on a single to right field, he’s not being aggressive enough. If you don’t ever get thrown out at third, you’re leaving runs on the table. I consider it the same way in terms of how quickly we implement new technologies and try and squeeze out a competitive advantage. If we’re not making some mistakes along the way, we’re not being aggressive enough.
If you wait for it to be obvious, it’s going to be too late. You have to be first.
Not to say human judgment is read out of the picture: To the contrary. Player attributes like leadership ability, simple desire, ability to overcome obstacles (resilience): These will always require human intuition and wisdom. But it’s the combination of the human factor with the analytics that’s the most powerful: “Either one separately gives you suboptimal results.”
Yes, changing people’s behavior on the field and how they interpret and use information and new technology “is very, very difficult to do. It’s been painful, and it’s taken a long time.” But once you’re there consider the sustainable distinction you’ve created.
“It’s going to provide us an advantage for the next five to ten years. […] It’s going to be hard for other clubs to copy that.”