Two New Books About Risk, Luck, and Skill Offer Insights For R&D Leaders

A central challenge of R&D, like many disciplines characterized by rare, outsized success, is how to think about risk, as well as the contributions of luck and skill. 

Two new books – How To Become Famous, by Cass Sunstein, and On the Edge, by Nate Silver, offer valuable perspectives. I’ll also highlight several articles that provide additional relevant insight, including two examples of conspicuous pharma failures.

How To Become Famous — Cass Sunstein

Cass Sunstein, a professor at Harvard Law School, is perhaps known for his interest in behavioral science, and is co-author, with Nobel Laureate Richard Thaler, of Nudge.

Sunstein’s more recent effort, How To Become Famous, examines success, and focuses on how remarkably contingent it can be. He acknowledges early on that the title is “a bit of a cheat” since “one of my main points is that there is no recipe for how to become famous… This is not a how-to manual.”

We learn, for example, about a fascinating study of authors of the Romanic period by University of Toronto Professor Heather Jackson, examining the trajectories of Wordsworth, Austen, Keats, and Blake on the one hand, and Crabbe, Southey, Cornwall, Hunt, and Brunton on the other. The names in the first group, but not the second, are now iconic, yet it wasn’t at all clear during their lives who would be remembered and who would be largely lost to history.

Jackson’s analysis, Sunstein says, “strongly suggests that accident, contingency, champions, and luck” played a “massive role” in guiding the outcome, rather than any difference in ability.  Roll the dice again and history might have come out differently.

Or consider music; we might believe that the most successful songs emerge from something intrinsically special about them.  Yet Sunstein presents data from an experiment called “Music Lab” revealing that preference for one (unfamiliar) tune over another can be driven by the putative relative popularity (which the experimenter can manipulate) of the two songs.  

“Everything turned on initial popularity,” Sunstein reports, and slight differences in initial preferences can play an outsized role in shaping the outcome. 

Success begets success, and popular songs tend to become even more popular.

Perhaps the most relevant lesson Sunstein has for biopharma executives is the fallacy of studies that select on the dependent variable. The typical examples here are approaches that examine outlier successes (prominent CEOs, startups, blockbuster drugs) and then extract common features, based on the idea these are critical factors presumably worthy of emulation. 

The problem, Sunstein says, is that this represents shoddy logic.  Such analyses, he argues, can offer “no idea whether the unifying characteristics were responsible for or even contribute to” the success. 

But he acknowledges that these narratives are hard to resist – hence the popularity (as Sunstein points out) of so many business books (and, I might add, consultant reports) premised on just this approach.   Everyone who has spent time in a pharma will recognize this pattern – generalizing on the “lessons learned” based on distinctive factors associated with an individual success. 

We might do well consider the insightful if uncomfortable reflections of former Pixar CEO Ed Catmull, who noted that there isn’t a template (or, in pharma parlance, a “playbook”) for a successful original film – you need to invent it anew each time.

Sunstein reminds us repeatedly about the ever-present role of randomness in success.  Consider, for example, the great (arguably the greatest) boxer Muhammad Ali.  When he (then Cassius Clay) was 12, Sunstein writes, growing up in Louisville, Kentucky, Clay’s bike was stolen, and he went to a police officer saying he wanted to “whup” the thief.  The officer he approached, Joe Martin, turned out to run a boxing gym, and told Clay that he might want to learn how to box.  He did.

Or consider the band Fleetwood Mac, best known for singer Stevie Nicks and guitarist Lindsey Buckingham. Founded in 1967 by drummer Mick Fleetwood, without either Nicks or Buckingham, it was a blues band and not very successful, Sunstein tells us. 

One day, when Fleetwood was evaluating a recording studio, an engineer happened to play a tape of Buckingham and Nicks (a pair of struggling musicians at the time) just to provide a sense of how the studio sounded.  A week later, one of Fleetwood’s guitarists left, and the band needed a replacement; they approached Buckingham, who agreed only if Nicks could join as well.   Before long, Fleetwood Mac’s dreams began to be realized.

Sunstein’s fundamental point is that “it’s a mistake to attribute spectacular success to the intrinsic qualities of those who succeed.  Of course, it is true that those who succeed may well be extraordinary…but their extraordinariness was hardly sufficient to get them where they ended up.  Countless extraordinary people never get very far.”

A member of the Obama administration from 2009-2012, Sunstein remembers a particularly salient observation made by the President about CEOs: “They’re lucky to be where they are.” 

Obama continued,

“They might be amazing, but still, they’re lucky to be where they are.  They got a lot of good breaks.  Some of them don’t seem to know that.  But it’s true.  Look at me.  I hope I’m doing a good job, but I had a lot of luck.”

On The Edge – Nate Silver

Best known for his political forecasts (he is founder of FiveThirtyEight, where he served as Editor-in-Chief until 2023), Nate Silver’s true passion, it turns out, is poker, which he played professionally before venturing into politics.

In his new book, On The Edge, Silver offers a glimpse into the mindset of what he sees as a distinct category of people who view life in terms of risk and probability, aspiring to identify situations where the math is in their favor (in their parlance: where the “expected value” or “EV” is positive). 

Silver refers to this “ecosystem” as the River, which includes not only gamblers, but also many investors (especially Silicon Valley VCs, with whom he seems especially enamored) and some philosophers. 

Silver contrasts Riverians with citizens of the Village, which consists generally of highly educated members of the media, government, and academia who he sees as comparatively risk adverse. 

Silver also believes many in the Village have an unhealthy tendency to “couple” – let their political beliefs intrude upon their analytics – rather than “decouple,” which he contends is “a type of intelligence that is valued in The River.” 

He describes decoupling operationally as the use of “yes, but” statements.  For example, he says, a Riverian might say: “Yes, I disagree with the Chick-fil-A CEO’s position on gay marriage, but they make a damn fine chicken sandwich.”  

Silver’s point is the speaker may or may not agree with the CEO’s politics but can separate in her head her view of the politics and her view of the food.   

While highlighting potential flaws within each camp, Silver proudly aligns himself with the River and describes a natural resonance with what he sees as their more probabilistic and contrarian view of the world.

Silver is clearly infatuated with poker and offers a level of detail reminiscent of Melville’s description of whaling.  Similarly, Silver’s extensive discussion of effective altruism and rationalism – not to mention Sam Bankman-Fried (Silver is not a fan, and considers him, in the words of Tim Wu, a “false prophet.) – will prove excessive for many readers (including this one).

Yet Silver’s ability to capture the mindset of Riverians feels relevant in R&D, a power law business driven by the exceedingly rare, outsized success – and where, I might add, key decisions are approached with a mindset that seems to combine extreme caution and faux mathematic precision.  It’s a domain where there’s an incredible premium on predicting success and “picking winners,” even though the actual ability of anyone to do this is arguably extremely low.

Of particular interest, Silver discusses how probabilistic thinking relates to two types approaches to knowledge. 

“The fox knows many things, but the hedgehog knows one big thing,” wrote the ancient Greek poet Archilochus, a formulation popularized in the mid-twentieth century by the philosopher Isaiah Berlin.  

A famous study by professor Philip Tetlock discovered (see this magnificent 2005 New Yorker review by Louis Menand), perhaps surprisingly, that the best predictors – “Superforecasters’ —  turned out to be foxes (particularly those with structured, probabilistic approaches), rather than hedgehogs.

Looking at Silicon Valley, Silver sees an ecosystem consisting of founders – hedgehogs – who tend to have a singular belief in the promise of their company, and foxes – VCs – whose job is to “herd hedgehogs,” assembling them into portfolios. 

Silver then invokes a useful poker analogy, envisioning a game where each turn, you need to either fold or go all-in.  He considers two strategies players might have – in one approach (“prudent”), the player decides based on her hand whether to fold or not.  In a second approach (“degenerate” or “degen”), the player goes all-in every single turn. 

What’s interesting here is that while the more cautious, arguably more skillful player does better on average, and winds up broke less often, the very highest scores are obtained by those who went all-in, even though on average, they do worse, and they go bankrupt most of the time. 

“Is this my way of saying that the richest founders in the world are just degenerate gamblers who got lucky?” Silver asks.  “No, I’m not saying that.  I think they’re highly skilled degenerate gamblers who got lucky.” 

Presumably, one might appropriately ask the same questions about the development of blockbuster drugs.  To give a particular candidate medicine the best chance of success, you need a highly skilled team – but ultimately, blockbusters likely require far more luck (and the resources to support the inevitable failures – see here) than the dialogue celebrating the rare success might suggest.

Four Articles Worth Reading

1. This thoughtful, concise read by Wharton Professor (and AI guru – see here) Ethan Mollick discusses how survivorship bias, together with compelling narratives, often lead to the confusion of skill and luck. His focus is entrepreneurship, but the argument applies readily to biopharma R&D as well.
2. A key paper cited by Mollick is this 2012 gem from Denrell and Liu, pointing out what savvy investors like Michael Mauboussin and Howard Marks have long recognized: in domains where luck plays a significant role in determining outcomes, the very highest performers are likely to have adopted an excessively risky strategy and happened to get lucky; in contrast, the most skilled performers are less likely to be at the very top most years, but will consistently do reasonably well.

Thus, seeking to emulate the characteristics of the top performers will not necessarily lead to the best results.  As the authors warn, “widespread use of this heuristic to identify whom to learn from can lead to diffusion of very risky behavior, and ‘nudges’ may be necessary to help people resist the temptation to praise, blame, or learn from extreme performers.”

A key point made by Nassim Taleb in Fooled by Randomness, and reinforced by Mollick, is that because of survivorship bias, it’s both essential and difficult to search out examples of failures and missed opportunities, to provide at least a measure of narrative balance. 

In this spirit, readers might consider:

3. This captivating, insider account by distinguished endocrinology researcher (and former Harvard Medical School Dean) Jeffrey Flier, describing an early effort to pursue several novel approaches to diabetes, including GLP-1, in conjunction with a biotech company (CalBio), and partnered with a large pharma (Pfizer). The collaboration was born in the 1990s.  

Concluding that injectable diabetes medicines (other than insulin) had no future – Pfizer abandoned the partnership. That led the biotech to abandon it as well. Notably, GLP-1s achieved blockbuster status for the treatment of diabetes even before the obesity indication was added.

4. This terrific STAT article by Jason Mast entitled “How Pfizer’s Grand Gene Therapy Ambitions Crumbled.” According to one Pfizer source with whom Mast spoke, “I think we promised probably too much without really understanding the limitations…We projected hope rather than reality.  We didn’t know.” 

One founder, Jude Samulski, explained that “We all expected it was going to be turnkey” – a standard delivery vehicle with different genes for different diseases.  He added “we were naïve in thinking that it was going to be a universal delivery system, in universally everybody, in universally every disease.”

Also recommended

Success and serendipity:

• This discussion of Seth Stephens-Davidowitz’s Don’t Trust Your Gut, focused in particular on how to increase exposure to positive serendipity — a “how to take action” component palpably absent from Sunstein’s analysis.
• This WSJ review of Malcolm Gladwell’s Outliers
• This discussion of Michael Mauboussin’s insightful perspective on luck and skill.
• This discussion of the role of luck and skill in film and pharma
• This 2008 Financial Times commentary about serendipity and pharma, co-authored with Nassim Taleb.

Narrative Bias:

• This discussion of narrative bias and a caution about tidy success stories.
• A discussion of VC Ali Tamaseb’s Superfounders, emphasizing the need to liberate founders from narrative bias.
• This discussion of the how heroic founder narratives, amplified by VCs and journalists, can adversely impact entrepreneurs.

Therapeutics Stories:

• This discussion of discovery of Keytruda.
• This discussion of the arduous development of GLP-1 medicines.
• This discussion of the early work on ibrutinib.
• This WSJ book review about the discovery and development of Botox.

Probability, forecasting, and longshots:

• This discussion of the fragility of pharma forecasts.
• This WSJ review of a new book discussing Bayes Theorem.
• This critique of “sterile information” associated with pharma forecasts.

The last word: Dr. Judah Folkman:

• This WSJ review of Safi Bahcall’s Loonshots. To quote from the last paragraph of the piece, “… it is often impossible for an organization to determine, in advance, whether an innovative team is working on a hit or a flop. As the late Harvard researcher Judah Folkman used to tell his students: ‘If your idea succeeds, everybody says you’re persistent. If it doesn’t, you’re obstinate.’”