Peter Thiel, the contrarian investor, had a favorite question for interviewees: “What important truth do few people agree with you on?”

My answer: No one can pick winners in pharma R&D. 

When I think of the most significant blockbusters in the industry involving novel mechanisms of action (follow-ons are a different story), I see a huge amount of luck on the exploration side, followed by exceptional focus on subsequent exploitation.

Pembrolizumab (Keytruda), Merck’s oncology blockbuster is one example (see here).  Millennium’s (later Takeda’s) bortezomib (Velcade) is another (here).  The GLP-1 medicines developed by Lilly and Novo and achieving stratospheric sales for the treatment of obesity are additional examples (see here, here). 

While the researchers leading the initial development of each of these medicines were passionate about the program’s prospects, each medicine had to overcome profound skepticism within a large pharma before it was eventually embraced and exalted. When Merck acquired Schering-Plough (and when Schering-Plough acquired Organon before that), the acquiring company had no idea a future blockbuster was lurking in its portfolio. Similarly, when Millennium acquired Leukosite, no value was attributed to the bortezomib program. 

In the case of Lilly and Novo, their work on incretins like GLP-1 represented an outgrowth of the longstanding interest in and involvement with diabetes in general, including the biology of insulin secretion (modulated by incretins). While early rodent studies of GLP-1 demonstrated an impact on appetite, diabetes — not obesity — was the initial focus.

When a leading incretin expert, Dr. Daniel Drucker, reviewed the field in 2017 – three years after the first GLP-1 was approved for obesity – he lamented the “market penetration” and worried about the “clinical appeal, expense, and commercial success of newer formulations” (see here).  While his concerns about expense were on target, these medicines have emerged as outsized successes, both clinically and commercially.   

The point is that while all these companies were transformed by those successful products, no one in R&D leadership at these companies saw this coming far in advance or had shaped their R&D strategies around the anticipation of such extraordinary results. 

What each company did do successfully was (eventually) recognize the promise of the medicine and work intensively to realize this potential.

The difficulty of replicating R&D success

One measure of the role of luck in R&D success is how difficult success is to replicate – how hard it is for companies to find their next blockbuster. 

My experiences suggest that in many ways, success makes finding the next blockbuster harder, because companies invariably develop a tidy narrative around the success – a comforting series of “lessons learned” (it was our culture, it was our focus on biomarkers, it was our clinical operations, etc), and then seek to impose these learnings on future company efforts. 

Yet if we’re in a domain where luck plays a critical role in determining success (spoiler alert: we are), then (as discussed here) you need to be really cautious about assuming that the occasional success represents the manifestation of your exceptional skill, rather than involving a combination of real skill coupled with a hefty amount of luck.  As Pixar’s Ed Catmull emphasizes, there isn’t a playbook (big pharma’s go-to crutch) for creative success like an original film (or, I’d add, an innovative drug) – you need to reinvent the approach each time.

There’s also an unfortunate tendency to fall prey to “selecting on the dependent variable,” as Cass Sunstein nicely explains in How To Become Famous (see here).  This approach, so familiar in companies and among the management consultants who advise them, involves looking at a set of successful companies/products/leaders and then trying to pull out what they have in common, and then concocting a compelling narrative around this.  Yet without knowing, rigorously, if these same traits are present among the many failures, you’re just coming up with a good story, with little actual validity or predictive value.

Are dart-throwing chimps the answer?

What is to be done?  Should the head of R&D at Lilly, Dan Skovronsky, acquire a trained chimp to throw darts at potential targets and programs, akin to the blindfolded monkey that economist Burton Malkiel famously asserted (in A Random Walk Down Wall Street) would pick stocks as well as expert investors?

At a fireside chat last Tuesday at Harvard Business School (where, disclosure, I serve as an advisor to the MS/MBA program in life sciences), I had an opportunity to see how Skovronsky viewed this exact challenge, as he responded to questions posed first by Professor Amitabh Chandra, then by members of the audience.

Refreshingly, Skovronsky emphasized the contingency of R&D, the role of chance, and the futility of commercial forecasts (a topic I’ve discussed frequently, see here, here, here).  He said (only half-jokingly) that the most important trait in a Chief Science Officer was being lucky.

He also emphasized that while some might urge Lilly to take its pile of GLP-1 cash and undertake either a transformative acquisition or acquire a derisked late-stage product, he wasn’t enthusiastic about either, noting that large M&A is profoundly disruptive, requiring massive corporate reorganizations which wasn’t his focus or interest.

Meanwhile, fully derisked products, he said, were typically acquired either by the company with the most excessive sales forecast (a concept referred to by economists as the “winner’s curse”) or by the company that is the most desperate.  Neither of these seemed like especially attractive prospects.

In contrast, Skovronsky suggested that a large pharma’s greatest strength is its ability to develop drugs, a notoriously complex task with which smaller organizations often struggle.

Interestingly, he suggested that because he was operating in an organization (and at a moment) where there was an uncommonly high tolerance for the inevitable failures, this represented a competitive advantage. 

Other pharmas, he suggested, might be less willing to invest in early, risky assets because leaders were worried about the career consequences if too many of these bets went south.

The notion of a large pharma arbitraging risk aversion (big if true…) seemed particularly interesting, and reminded me of PayPal’s ability to take advantage of large company risk aversion (specifically in the context of regulatory risk) to outmaneuver the combination of eBay plus Bank of America – see here.

Skovronsky also suggested that because of its location in Indianapolis, Lilly was somewhat protected from the latest hype cycles and had relatively more patience to pursue promising but difficult programs. 

Perhaps related, he also offered a decidedly reserved perspective on AI, allowing there was promise in assisting with molecule creation, but otherwise fairly guarded about the impact.

I hope Skovronsky’s approach succeeds, though it’s difficult not to look at Moderna as a concerning example. Moderna may have slipped silently into obscurity were it not for the opportunity afforded by COVID, an incredibly fortuitous (from the perspective of the company at least) event for which it was ideally positioned. 

Of course, the success narratives soon followed – it was their embrace of digital, or their adoption of AI, or their culture, that enabled them to be successful, and would surely pave the road for future success.  Betting on themselves, they invested significantly in R&D.

Unfortunately, these efforts, to date, haven’t panned out; their stock has been crushed by investors, and after much outcry from analysts, Moderna recently announced massive R&D cuts.  Mr. Market can be very unforgiving.

I hope this isn’t Lilly’s future, and I also hope that Moderna’s pipeline and approach yields future impactful medicines. But whether we should count on this happening is a more difficult question.

Contingent, but not Random

If, as I believe, there isn’t a leader of R&D on the planet who has a magical or uncanny ability to “pick winners,” how do you approach this role?  What role does good luck play?  Is it time to bring in the blindfolded chimp?

Before we answer this, we should come to terms with perhaps a second truth about R&D that can be difficult for those outside the process to appreciate: R&D programs are not discrete, static, or hermetic – it’s not like betting that a roulette wheel lands on “36.” 

Rather, drug development requires constant effort to perfect and refine a new medicine, at every stage of the process including in clinical trials, where the right patient population and dosage scheme could make the difference between a failed and successful product. 

The most significant challenge facing any head of R&D is balancing the recognition that, on the one hand, outsized success is generally impossible to predict in advance, and most projects involving novel mechanisms will fail.  On the other hand, for any project to have a chance at success, it must proceed with full and authentic conviction, with leaders absolutely convinced that there’s a path for success to be found.

Balancing these two is incredibly challenging, particularly the need to believe deeply in programs while somehow compartmentalizing enough to avoiding getting sucked entirely into the program’s narrative, and maintaining enough objectivity to make difficult, data-driven decisions (see also here).

Sometimes, as Skovronsky points out, the issue isn’t data but politics; as previously reported in the Wall Street Journal, he described how Lilly (and other pharmas) historically, would advance programs with shaky data because the program would fulfill an identified commercial need (a concern VC David Grainger has raised as well, as I’ve discussed here).

Since these ill-advised programs would generally fail in subsequent, more expensive later phase clinical studies, they are huge wastes of time and resources, and Skovronsky has worked aggressively to eliminate them.

But in many (arguably most) cases, it’s less clear whether a program is six months away from a breakthrough (as teams tend to argue) or six months away from just digging a deeper hole.  Management gurus will tell you that the ability to discern one from the other is the secret of great leaders. This is wishful thinking. 

Many top scientists, like famed Harvard surgeon-researcher Judah Folkman, acknowledge that when you’re in the midst of an impassioned pursuit, you have no way of knowing whether you are persistent or obstinate; that distinction is revealed only by the outcome. 

Given the need for R&D leaders to come to terms with the inherent uncertainty associated with their roles, they would do well to remove sterile information where possible. In a futile effort to domesticate uncertainty, many R&D organizations, especially when things seem to be going poorly, decide they will improve decision making by imposing stricter discipline, and start affixing all sorts of quantification (e.g. probability of technical success, estimated net present value [NPV]) across R&D. These tend to be falsely precise numbers that typically provide far more illusory comfort than true insight. 

I would love to see the NPVs that Merck associated with pembrolizumab when it acquired Schering-Plough, that Millenium associated with bortezomib when it acquired Leukosite, or that Novo and Lilly associated with obesity indications of GLP-1 programs during their early days (or even their not-so-early days, as the Drucker experience suggests).

Bottom Line

The fundamental challenge for R&D leaders is that while success is highly contingent, your only way to achieve it is through convicted pursuit, though such pursuit is not (at all) a guarantee of success. The ability to muster conviction in the face of inherent overwhelming uncertainty is the critical quality required for R&D leaders and others who seek to leave their mark on the world.

A central challenge of R&D, like many disciplines characterized by rare, outsized success, is how to think about risk, as well as the contributions of luck and skill. 

Two new books – How To Become Famous, by Cass Sunstein, and On the Edge, by Nate Silver, offer valuable perspectives. I’ll also highlight several articles that provide additional relevant insight, including two examples of conspicuous pharma failures.

How To Become Famous — Cass Sunstein

Cass Sunstein, a professor at Harvard Law School, is perhaps known for his interest in behavioral science, and is co-author, with Nobel Laureate Richard Thaler, of Nudge.

Sunstein’s more recent effort, How To Become Famous, examines success, and focuses on how remarkably contingent it can be. He acknowledges early on that the title is “a bit of a cheat” since “one of my main points is that there is no recipe for how to become famous… This is not a how-to manual.”

We learn, for example, about a fascinating study of authors of the Romanic period by University of Toronto Professor Heather Jackson, examining the trajectories of Wordsworth, Austen, Keats, and Blake on the one hand, and Crabbe, Southey, Cornwall, Hunt, and Brunton on the other. The names in the first group, but not the second, are now iconic, yet it wasn’t at all clear during their lives who would be remembered and who would be largely lost to history.

Jackson’s analysis, Sunstein says, “strongly suggests that accident, contingency, champions, and luck” played a “massive role” in guiding the outcome, rather than any difference in ability.  Roll the dice again and history might have come out differently.

Or consider music; we might believe that the most successful songs emerge from something intrinsically special about them.  Yet Sunstein presents data from an experiment called “Music Lab” revealing that preference for one (unfamiliar) tune over another can be driven by the putative relative popularity (which the experimenter can manipulate) of the two songs.  

“Everything turned on initial popularity,” Sunstein reports, and slight differences in initial preferences can play an outsized role in shaping the outcome. 

Success begets success, and popular songs tend to become even more popular.

Perhaps the most relevant lesson Sunstein has for biopharma executives is the fallacy of studies that select on the dependent variable. The typical examples here are approaches that examine outlier successes (prominent CEOs, startups, blockbuster drugs) and then extract common features, based on the idea these are critical factors presumably worthy of emulation. 

The problem, Sunstein says, is that this represents shoddy logic.  Such analyses, he argues, can offer “no idea whether the unifying characteristics were responsible for or even contribute to” the success. 

But he acknowledges that these narratives are hard to resist – hence the popularity (as Sunstein points out) of so many business books (and, I might add, consultant reports) premised on just this approach.   Everyone who has spent time in a pharma will recognize this pattern – generalizing on the “lessons learned” based on distinctive factors associated with an individual success. 

We might do well consider the insightful if uncomfortable reflections of former Pixar CEO Ed Catmull, who noted that there isn’t a template (or, in pharma parlance, a “playbook”) for a successful original film – you need to invent it anew each time.

Sunstein reminds us repeatedly about the ever-present role of randomness in success.  Consider, for example, the great (arguably the greatest) boxer Muhammad Ali.  When he (then Cassius Clay) was 12, Sunstein writes, growing up in Louisville, Kentucky, Clay’s bike was stolen, and he went to a police officer saying he wanted to “whup” the thief.  The officer he approached, Joe Martin, turned out to run a boxing gym, and told Clay that he might want to learn how to box.  He did.

Or consider the band Fleetwood Mac, best known for singer Stevie Nicks and guitarist Lindsey Buckingham. Founded in 1967 by drummer Mick Fleetwood, without either Nicks or Buckingham, it was a blues band and not very successful, Sunstein tells us. 

One day, when Fleetwood was evaluating a recording studio, an engineer happened to play a tape of Buckingham and Nicks (a pair of struggling musicians at the time) just to provide a sense of how the studio sounded.  A week later, one of Fleetwood’s guitarists left, and the band needed a replacement; they approached Buckingham, who agreed only if Nicks could join as well.   Before long, Fleetwood Mac’s dreams began to be realized.

Sunstein’s fundamental point is that “it’s a mistake to attribute spectacular success to the intrinsic qualities of those who succeed.  Of course, it is true that those who succeed may well be extraordinary…but their extraordinariness was hardly sufficient to get them where they ended up.  Countless extraordinary people never get very far.”

A member of the Obama administration from 2009-2012, Sunstein remembers a particularly salient observation made by the President about CEOs: “They’re lucky to be where they are.” 

Obama continued,

“They might be amazing, but still, they’re lucky to be where they are.  They got a lot of good breaks.  Some of them don’t seem to know that.  But it’s true.  Look at me.  I hope I’m doing a good job, but I had a lot of luck.”

On The Edge – Nate Silver

Best known for his political forecasts (he is founder of FiveThirtyEight, where he served as Editor-in-Chief until 2023), Nate Silver’s true passion, it turns out, is poker, which he played professionally before venturing into politics.

In his new book, On The Edge, Silver offers a glimpse into the mindset of what he sees as a distinct category of people who view life in terms of risk and probability, aspiring to identify situations where the math is in their favor (in their parlance: where the “expected value” or “EV” is positive). 

Silver refers to this “ecosystem” as the River, which includes not only gamblers, but also many investors (especially Silicon Valley VCs, with whom he seems especially enamored) and some philosophers. 

Silver contrasts Riverians with citizens of the Village, which consists generally of highly educated members of the media, government, and academia who he sees as comparatively risk adverse. 

Silver also believes many in the Village have an unhealthy tendency to “couple” – let their political beliefs intrude upon their analytics – rather than “decouple,” which he contends is “a type of intelligence that is valued in The River.” 

He describes decoupling operationally as the use of “yes, but” statements.  For example, he says, a Riverian might say: “Yes, I disagree with the Chick-fil-A CEO’s position on gay marriage, but they make a damn fine chicken sandwich.”  

Silver’s point is the speaker may or may not agree with the CEO’s politics but can separate in her head her view of the politics and her view of the food.   

While highlighting potential flaws within each camp, Silver proudly aligns himself with the River and describes a natural resonance with what he sees as their more probabilistic and contrarian view of the world.

Silver is clearly infatuated with poker and offers a level of detail reminiscent of Melville’s description of whaling.  Similarly, Silver’s extensive discussion of effective altruism and rationalism – not to mention Sam Bankman-Fried (Silver is not a fan, and considers him, in the words of Tim Wu, a “false prophet.) – will prove excessive for many readers (including this one).

Yet Silver’s ability to capture the mindset of Riverians feels relevant in R&D, a power law business driven by the exceedingly rare, outsized success – and where, I might add, key decisions are approached with a mindset that seems to combine extreme caution and faux mathematic precision.  It’s a domain where there’s an incredible premium on predicting success and “picking winners,” even though the actual ability of anyone to do this is arguably extremely low.

Of particular interest, Silver discusses how probabilistic thinking relates to two types approaches to knowledge. 

“The fox knows many things, but the hedgehog knows one big thing,” wrote the ancient Greek poet Archilochus, a formulation popularized in the mid-twentieth century by the philosopher Isaiah Berlin.  

A famous study by professor Philip Tetlock discovered (see this magnificent 2005 New Yorker review by Louis Menand), perhaps surprisingly, that the best predictors – “Superforecasters’ —  turned out to be foxes (particularly those with structured, probabilistic approaches), rather than hedgehogs.

Looking at Silicon Valley, Silver sees an ecosystem consisting of founders – hedgehogs – who tend to have a singular belief in the promise of their company, and foxes – VCs – whose job is to “herd hedgehogs,” assembling them into portfolios. 

Silver then invokes a useful poker analogy, envisioning a game where each turn, you need to either fold or go all-in.  He considers two strategies players might have – in one approach (“prudent”), the player decides based on her hand whether to fold or not.  In a second approach (“degenerate” or “degen”), the player goes all-in every single turn. 

What’s interesting here is that while the more cautious, arguably more skillful player does better on average, and winds up broke less often, the very highest scores are obtained by those who went all-in, even though on average, they do worse, and they go bankrupt most of the time. 

“Is this my way of saying that the richest founders in the world are just degenerate gamblers who got lucky?” Silver asks.  “No, I’m not saying that.  I think they’re highly skilled degenerate gamblers who got lucky.” 

Presumably, one might appropriately ask the same questions about the development of blockbuster drugs.  To give a particular candidate medicine the best chance of success, you need a highly skilled team – but ultimately, blockbusters likely require far more luck (and the resources to support the inevitable failures – see here) than the dialogue celebrating the rare success might suggest.

Four Articles Worth Reading

1. This thoughtful, concise read by Wharton Professor (and AI guru – see here) Ethan Mollick discusses how survivorship bias, together with compelling narratives, often lead to the confusion of skill and luck. His focus is entrepreneurship, but the argument applies readily to biopharma R&D as well.
2. A key paper cited by Mollick is this 2012 gem from Denrell and Liu, pointing out what savvy investors like Michael Mauboussin and Howard Marks have long recognized: in domains where luck plays a significant role in determining outcomes, the very highest performers are likely to have adopted an excessively risky strategy and happened to get lucky; in contrast, the most skilled performers are less likely to be at the very top most years, but will consistently do reasonably well.

Thus, seeking to emulate the characteristics of the top performers will not necessarily lead to the best results.  As the authors warn, “widespread use of this heuristic to identify whom to learn from can lead to diffusion of very risky behavior, and ‘nudges’ may be necessary to help people resist the temptation to praise, blame, or learn from extreme performers.”

A key point made by Nassim Taleb in Fooled by Randomness, and reinforced by Mollick, is that because of survivorship bias, it’s both essential and difficult to search out examples of failures and missed opportunities, to provide at least a measure of narrative balance. 

In this spirit, readers might consider:

3. This captivating, insider account by distinguished endocrinology researcher (and former Harvard Medical School Dean) Jeffrey Flier, describing an early effort to pursue several novel approaches to diabetes, including GLP-1, in conjunction with a biotech company (CalBio), and partnered with a large pharma (Pfizer). The collaboration was born in the 1990s.  

Concluding that injectable diabetes medicines (other than insulin) had no future – Pfizer abandoned the partnership. That led the biotech to abandon it as well. Notably, GLP-1s achieved blockbuster status for the treatment of diabetes even before the obesity indication was added.

4. This terrific STAT article by Jason Mast entitled “How Pfizer’s Grand Gene Therapy Ambitions Crumbled.” According to one Pfizer source with whom Mast spoke, “I think we promised probably too much without really understanding the limitations…We projected hope rather than reality.  We didn’t know.” 

One founder, Jude Samulski, explained that “We all expected it was going to be turnkey” – a standard delivery vehicle with different genes for different diseases.  He added “we were naïve in thinking that it was going to be a universal delivery system, in universally everybody, in universally every disease.”

Also recommended

Success and serendipity:

• This discussion of Seth Stephens-Davidowitz’s Don’t Trust Your Gut, focused in particular on how to increase exposure to positive serendipity — a “how to take action” component palpably absent from Sunstein’s analysis.
• This WSJ review of Malcolm Gladwell’s Outliers
• This discussion of Michael Mauboussin’s insightful perspective on luck and skill.
• This discussion of the role of luck and skill in film and pharma
• This 2008 Financial Times commentary about serendipity and pharma, co-authored with Nassim Taleb.

Narrative Bias:

• This discussion of narrative bias and a caution about tidy success stories.
• A discussion of VC Ali Tamaseb’s Superfounders, emphasizing the need to liberate founders from narrative bias.
• This discussion of the how heroic founder narratives, amplified by VCs and journalists, can adversely impact entrepreneurs.

Therapeutics Stories:

• This discussion of discovery of Keytruda.
• This discussion of the arduous development of GLP-1 medicines.
• This discussion of the early work on ibrutinib.
• This WSJ book review about the discovery and development of Botox.

Probability, forecasting, and longshots:

• This discussion of the fragility of pharma forecasts.
• This WSJ review of a new book discussing Bayes Theorem.
• This critique of “sterile information” associated with pharma forecasts.

The last word: Dr. Judah Folkman:

• This WSJ review of Safi Bahcall’s Loonshots. To quote from the last paragraph of the piece, “… it is often impossible for an organization to determine, in advance, whether an innovative team is working on a hit or a flop. As the late Harvard researcher Judah Folkman used to tell his students: ‘If your idea succeeds, everybody says you’re persistent. If it doesn’t, you’re obstinate.’”