The article du jour on this topic is “Don’t Bury Big Law Just Yet” (whence the “alive, well, and rich” line). One correspondent characterized this piece to his readers as “It is official. Everything is just fine in BigLaw. The American Lawyer just told us so.” Here’s the lead from the piece and a bit more (emphasis mine):
Headlines like “The Last Days of Big Law” are great—eye-catching and search engine–optimized. If only they were true. While it’s always easy to use “anecdata” and argue, loosely, that the fate of one individual firm, say, the late Dewey & LeBoeuf, is a harbinger for the industry as a whole, that’s actually not the case when it comes to big law firms.
[…] A deeper look at how the largest firms are doing, short-term and long-term, shows resilience and growth.Take the latest results from The Am Law 100 for 2012. The 100 biggest law firms, ranked by their gross revenue, continued to post revenue and profit gains even as the U.S. economy sputtered.
[…]As my colleague Chris Johnson pointed out, some $2.4 billion was wiped from The Am Law 200’s collective top lines in 2009—the first fall in overall revenue since we started tracking law firm financials more than two decades ago. But that amount was recouped within 12 months, with total group revenue rising $3 billion in 2010 and continuing to grow at a CAGR of 4.1 percent over the past two years.
Finally, there’s this:
History shows exactly how durable this industry is. In 2012, as part of our 25th anniversary coverage of The Am Law 100, we looked at the firms that appeared on our first ranking in 1986 and compared it to the 2012 roster. Sixty-nine firms appeared on both lists, while 12 firms moved to the Second Hundred. The group of 81 firms generated $6 billion in revenue in 1986 and $58 billion in 2011.
[…] But make no mistake, the giant law firms in this country are alive, well, and rich. Rumors of their demise are greatly exaggerated.
Shall we take “a deeper look,” as we’ve been invited to do, and see what we find?
Let’s start with the “history shows” paragraph. Here’s an example of survivorship bias elevated to doctrinal encyclical. We can stipulate that 69 of the original AmLaw 100 firms remained there 25 years later (or 81 of what would have been the AmLaw 200?—no, that’s not quite right, is it?—hard to tell on the information given) but what of the other 31, call it one-third of the universe in question?
- Did they fail?
- Did they merge with other AmLaw 100/200 firms?
- Did they just slide into unranked purgatory?
The methodological flaw is one you could drive a bus full of credulous AmLaw partners through, and introducing the error of survivorship bias is so widely recognized in statistical analysis that it has its own Wikipedia entry: “Survivorship bias is the logical error of concentrating on the people or things that “survived” some process and inadvertently overlooking those that didn’t because of their lack of visibility. This can lead to false conclusions in several different ways.”
More precisely for present purposes “In finance, survivorship bias is the tendency for failed companies to be excluded from performance studies because they no longer exist. It often causes the results of studies to skew higher because only companies which were successful enough to survive until the end of the period are included.”
In short, the “$6 billion/$58 billion” numbers, while they look like data, are anything but. It’s hard to know what they mean beyond, more or less, “firms that survived and remained at the top for 25 years survived and remained at the top.”
But this part is too easy. Of far greater interest is to dig into those CAGR numbers and see where they lead.
