On this week’s show, Matt and Nick meet Chengwei Liu, professor of strategy and behavioural science at Imperial College, London, to discuss the role of luck in decision making — and why we routinely underestimate it.
Chengwei traces his own serendipitous path from consulting in Singapore to a PhD at Cambridge, and explains why so much of what gets credited to skill in business is really survivorship bias dressed up after the fact. The conversation gets into the case for random selection: when shortlisted candidates or options are genuinely close, drawing lots produces outcomes equivalent to deliberation while avoiding the bias and politics that creep in when committees feel obliged to justify a winner. Switzerland has been allocating research grants this way for over a decade, and the UK’s British Academy is now following.
The discussion also picks up on the cultural and psychological complications — how randomness reads as fate or destiny in some cultures, why women may be more likely to internalise an unlucky outcome as personal failure, and whether modern corporate life can ever sit comfortably with “we rolled a dice” as a legitimate answer. Plus, a look ahead to Chengwei’s forthcoming book, Smart Contrarians: How to Think Differently About Thinking Differently.
Transcript automatically generated by Descript
Matt: Hello and welcome to episode 347 of WB 40, the increasingly erratic podcast. With me, Matt Ballantine, Nick Drage, and Chengwei Liu.
Matt: Welcome back. We have been away for a few weeks. There was the Easter break and, uh, the usual weird way that everything seems to cease for a while because of a couple of days of bank holiday. And, uh, we are back again. And, um, just before we started recording, uh, Nick. Described himself as being the robin to my Batman, which now means all I’ve got is mental images of Nick wearing tights, which probably I don’t need.
Nick, how has your last week or so been.
Nick: Well, I mean, I’m instantly regretting that version and not knowing more about is the DC comics background? Is it, isn’t it? And so on there, there must been, must be a much more, less sixties robin. Than what came to our minds apart. The main thing in the previous week is attending Connections online, which is a, an online war gaming conference based in essentially eastern US time zones and just a chance to catch up with a lot of thinking, a lot of people.
And they used the online version to really get into. Non-standard ideas such as how to model soft factors in war games, and LLMs of course featured quite heavily. So that was really good the previous week. How about you?
Matt: So what have I been doing here the last week? Had a wonderful lunch with a fascinating academic last Monday. That was good. Hi Chen Way. And then Tuesday was the first run of random, the workshop in what I think is as close to being its final form as ever. It will ever get into a final form.
So I had, despite the tube strikes in London, I had a dozen people join me at the Equal Experts offices in London. And we had. A really interesting evening for me to be able to try out some of the ideas of how to get people to explore the concepts and the ideas that we’ve got within the book that is coming out now.
And we have a we have now a publication date, which is, or launch date, the 17th of June, which is extremely exciting. So that was good. I’ve had some time to be able to, just make some decisions as well. I was running with my foot in two camps at work for a while ’cause we’ve had a bit of a restructure and just because of the nature of the clients that I’ve got, I ended up in sort of two parts of it.
And so last week I also took the executive decision to be able to say, I’m going to focus on one, not try to continue to be in both. And that’s really reduced my stress levels immeasurably. There’s something about, being able to, it’s not so much about resolving ambiguity as actually when you’ve got too many things that you could be doing and needing to be able to make a decision about being able to focus on things that you can manage to do and how being able to turn off the fomo, the fear of missing out, and instead be able to properly commit to doing a manageable set of things.
I think that, that’s been a good learning. And then apart from that, obviously I’ve been obsessively watching marine tracking apps to be able to see where the boat on which the book is sitting. And I had a sort of nervous few days because. It’s been in the middle of the Indian Ocean and as a result there’s been no tracking updates.
But it came back onto the the system again about 14 hours ago. So it’s just heading past Madagascar now, and it’s been a fascinating insight into what it actually means for goods to be manufactured in China, and then the route that they take to be able to make it. To the UK and you, this is stuff that we don’t think about and it’s been really interesting to be able to just see how this stuff sits together.
So that’s been my week, I think. So with that welcome to the show Chengwei what’s your week been like other than a wonderful lunch with this strange man that you met on last Monday?
Chengwei: Hello Matt and Nick thank you for having me. Yeah, my week was definitely great highlighted by the excellent lunch with you. We talk all ideas, right? And really glad to learn that your workshop went well because I have the pleasure of having you as my guest speaker for my MBA students at the Imperial College in a couple of months as well.
And the rest of week I managed to finish the revision of my new book. Smart contrarians, how to think differently about thinking differently. So I’m really look forward to the next stage. That is the marketing stage of this book.
Matt: So whereabouts are you now? Is everything is with publisher? Has it been printed yet or is it still at the place where it’s.
Chengwei: No, it’s the handling editor at hover Business Review Press, which is our publisher. They sign it off. The editor and his boss sign it off, so it’s. Good from their perspective. But then the first stage in the production is there will be a copy editor go through the entire book, and then of course suggest questions and also some fact checking, right?
So if we have any say missed and then they’ll point it out and we just verify everything it’s in the right place before they put it in print.
Matt: It is interesting, isn’t it, when you actually get into the process of producing a book. I, for many years wondered why it took so long from somebody saying, I finished writing it to it actually being available. And then when you actually go through the process, you start to realize why. ’cause there’s an awful lot of work involved in producing a book.
Chengwei: Absolutely that the divisional labor there is very interesting. I begin to learn about this as well, and there’s a meeting event with the market department of the Harvard Publishing. That’s a huge group actually in the US And yeah, they have their expertise and we just have this interesting.
Serious about deciding the book cover. So basically the first cover that designers sent were really not so happy with it, but our headly editor said, okay, there’s not a thing he can really intervene too much. And, but we, he gave us. Some ideas about how to say nicely that we hate it and we did it.
And then and of course we provided our view about what the cover should, like what is the concept, and of course, the taboo to say anything specific about design idea, because that’s their domain, right? We don’t want to invade their domain with the academic author. Author of the book we control or have the ownership of the ideas and the concept being written.
But our design cover is their domain. We can provide ideas, suggestion, but they decide. So fortunately the second cover idea they send where. Reasonably happy. So that is another major achievement in the past week. Now, I think about it because initially when we received and we thought about the previous discussion with the editor, then we really worry if that is something we need to accept.
We are not really, be comfortable with that.
Matt: It’s interesting for you as somebody who has studied a lot about decision making, a aesthetic decisions like that is something, is that something that you approach differently from, I dunno, more, more rational things? ’cause it is, that’s all, there’s an awful lot about gut feel and about things that are quite intangible with something like, I like a book cover or not.
Chengwei: Yes, exactly. That’s very interesting. I think it’s a contrast between the art designer. In the email she sent, she actually provide a lot of rational justification why the first cover design was the brilliant idea and saying that the huge contrast in terms of color and the ice, the idea she used was.
The ice like all closed and one kind of open, that is a contrarian idea, right? When the i where the crowd is not thinking or is just a blind, then you have the idea. But the I idea is a little bit too ag, so we don’t like it. But the way she frame it is that based on her experience about the market and how your book will appear on the bookshelf in the airport bookstore, for example, that really.
There now, then attract attention. And that’s what you need, right? Occupying the attention. And in contract, usually we’re more comfortable. My call and I, we teach decision making, so we’re comfortable in the domain. We are, like, I just talked to Nick about, we teach business strategy, merger acquisition, right?
So all this kind of thing. In those domain, we can be very rational and to reduce this overconfidence whatsoever. But in the publishing world, we are really the inexperienced one. We don’t know what to things, so we can only rely on our gut feeling about, okay, it looks. Not right to us and we want to push back, but probably she’s right about something.
And eventually, of course, the second design have, still have the idea but also remove the eyes and use some other things, some other symbol. So we’re happy with that. But one of the thing of course I use is not that feeling, but my wife’s got feeling. She feel very happy with the second design. Then that’s all I need.
Then it can go for it. I can, we can go for it.
Matt: That’s brilliant. We’re gonna be talking a lot more about questions of decision making of. Luck, serendipity, and some of the stuff around contrarians as well over the rest of this show. So I think we should probably crack on.
The reason you and I have been, have got to know each other a bit recently, Chengwei, is because we’ve got a a mutual contact mark and mark said that I needed to meet you because I don’t think it, it quite described this, but basically there’s this professor of Luck that I know. And so it sounded like a fascinating thing to talk to you about.
And it, it has. Proven so, but let’s, if we could start by just exploring that a bit, a little bit, because some of your academic work, which is big, serious economics, social psychology, behavioral science, those kind of areas but you have looked at the role that luck has to play within the realm of decision making.
So what drew you to that in the first place?
Chengwei: Yeah. Yeah, that’s the brilliant randomness, right? The journey is actually pretty serendipitous. How I enter this research topic and all how I met our mutual contact mark, it’s all about luck ’cause you are in my after my brief introduction of my intellectual journey. So I was a consultant.
I, I’m from Taiwan originally. When I graduated, I found a job as a consultant in Singapore. And then saw the experience I had really make me wonder. And of course, I was trained as an economist. I tried to do all this rational analysis, very formal analysis, recommending like the optimal pricing strategy, optimal strategy in general to our clients.
Of course I was a junior, so I did most of the analysis, the hard work, and then the senior partner tried to sell all these ideas to the client, right? And sometimes I observe. And the client actually did not follow what we recommended, even with all this kind of backing with the data and a analytics.
But they still perform very well to our surprise, or to their own surprise as well. And sometimes they follow our recommendation, but the performance entirely shocking, shockingly bad. So that made me wonder, okay, so there may be a little bit of what the rational decision making analysis can do, right?
So you can make. The best possible decision in terms of decision making process, but still it’s a terrible outcome and vice versa because of all this lack of foresight, lack of unpredic lack of predictability. So that kind of motivate me to think about this. Wow. Isn’t anyone in the world studying this perspective of business strategy back then?
And then I met a professor at National University of Singapore, who’s from Taiwan as well. He’s my mentor and he introduced me to. His classmate when he studied his PhD in the uk and then was and then this professor was at Cambridge. He wrote a brilliant article in a very good journal in our field called CH Choice and Ineffability in Strategy.
So that’s actually a very interesting philosophical take of business strategy that you will not anticipate. But that’s so essential if you think about it, because that’s, that’s related to the philosophical debate about the scope of free will, right? So it’s, everything is predetermined, is everything inevitable?
Actually, the strategizing we talk about, just like a self construction is not really any scope for decision making, right? But then in, in that article, he carefully argue and carefully develop argument. Yes, a lot of things are predetermined due to past dependency. But still there’s some scope for strategizing for serendipity because it’s not entirely determining.
It’s a little bit random operating there. So I was fascinated by this idea. Then I quit my consultanting job and joined a PhD without realizing actually you’ll have no income for the next five years. And so that was a little bit shocking. I was very lucky in the sense that because I followed my wife to some extent, I wanted to do a PhD in the us.
I always wanted. But this topic choice and the location choice in the uk, actually, I follow my wife. She got a. A master degree in Finance first in, in, in the uk, in London actually. And then to everyone’s surprise, she found a job just during the financial crisis, a very good job. And then, okay then it’s almost impossible to move to the us.
I find a job there. So I just follow her and come to the UK and did my PhD in Cambridge Whisk. The supervision of that particular professor, my kind of mentors classmate the professor I met in Singapore. So that is the serendipitous part, right? So I could not have predicted any of that.
But then it happened so. However, then I became the black sheet in Cambridge or in the business school because my advisor the supervisor in Cambridge, he’s a hardcore ethnographer. He really did this very in-depth case study and follow the subject he studied, organization, he studied in a very qualitative way.
Of course I tried that a little bit from the beginning. I did I follow many of the successful entrepreneurs and manager in the Cambridge ecosystem and something in London, and I document many fascinating story, but then to publish or write out my dissertation the very usual feedback I got is that okay?
You get another fascinating story. But so what? Because most of the story I heard is that, okay, so entrepreneur or engineer in Cambridge are work on something very nice and had some pattern but don’t know how to commercialize until he or she met business Angel in one of the networking event.
And they did not plan to go, but just happened to drag by a friend and met this life-changing person, become a business partner. So the structure, all the story look very much like that. There were some story, or sorry, some research in sociology. Or network science already did about that. The strengths of weak ties, for example.
You need to engage some of this less frequently contacted people, friends or the acquaintance of your acquaintance so that they don’t have this overlap information or they have this non-overlapping expertise, right? That’s more likely to get lucky from that perspective. But then in terms of PhD dissertation, that is not good enough.
There’s no new theoretical breakthrough. So that’s why I changed my course during my second year with the blessing of my supervisor. So I’m really grateful to his. Like mentorship, gimme the intellectual freedom because I entirely changed the methodology and the focus, and I become the computational researcher.
So I basically use model to formalize. When the outcome is more likely to be a matter of randomness induced by kind of randomness and how interact with the environment instead of the skill or the effort of the individuals or the organization. And then also use experiment to test when people can systematically.
Mistake, love for skill, like overestimating all these outcomes or successes to the person involved instead of the situation they’re in, then that’s it. That is the beginning of my kind of academic career. I was very fortunate to publish a few paper at the beginning of my career.
Kind of set the foundation of the next 15 years of my career. So most all my research can be tracked to this idea about mistaking love for skill. And what can you do about that?
Matt: And in terms of the kind of overall headlines of what you found through that work is it possible to put proportions onto how often it is that really you, if not everything is to do with luck? There’s obviously, hard work, skill, talent, and luck all combine, but.
When people talk about being self-made, for example, I saw a thing on LinkedIn. A few Forbes seemed to be just churning out lots and lots of top one hundreds and top twenties of all sorts of things you couldn’t imagine. One of them is the top 100 self-made people. And if you were to look at the list of self-made, obviously they don’t mean self-made.
They mean have been able to be able to use the work of others to be able to be successful. But. Are any of these people immune to it having been a lucky break somewhere along the line?
Chengwei: Yeah, absolutely. In my own book about luck actually there’s a chapter about how to quantify luck and that’s exactly solving this very important question, right? Of course, that is not the that is not the math that can really give you a very clear percentage for individual event.
Or individual success, whether how much is the can be attributed to skill and effort, how much to being at the right place at the right time, but in a domain like all the musicians that who, who are on the Billboard a hundred list or, industry, we can have some idea about how much log involves in a particular performance range.
So if I take one step back, I think the main takeaway from my exercise about this quantifying luck is that the more successful an individual firm is, the more the role the more important the role of luck is particularly for the exceptional. Exceptionally successful. You talk about the top 100 self-made people, they’re much more likely than the, I would call the second best, the successful, but not exceptionally successful people there.
The circumstances play in the former’s role is, much greater. So let me give you one example and also to illustrate this methodology I created. So the three best sellers in management is in search of excellence that published in the eighties last century. And then there was a book called Good to Great.
By co. There’s a, oh, at some point oversell even the Bible in the us. Very popular. And then he had a sequel called Built To Last as well. So these are the three best seller and the methodology this author use our very straightforward and really talk to our gut feeling they collected the most successful firms or entrepreneurs during their time.
And study what they did. All this firm and the individuals. And then the common feature or the common practices, they just frame them as the best practices. So if you follow this best practices, presumably you can move from good to great deal to last then become excellent. So this is a formula of this best salary management, right?
And that’s how they just sell millions of copy. One analysis I did is I analyzed what happened to those firms after the book was published, and now you can probably see where this is going, right? One third of the firm among the three books about 50 firms being featured, one third amount them when bankrupted within five years of the publication of the book and the rest.
Underperform, even like index fund, like s and p 500, like every performance for large cooperation in the us, right? So what happened after this exceptional success is regression to the mink. Not just huge or regression to the ME because every performance regression to the ME because performance is a combination of skill and some random component, right?
So high performance, or probably a high dose of skill and strategy, good strategy, but also high dose of luck. And by definition, luck is not persistent. So very high performance is going to be followed by. Less high performance in the future, regressing downward to the me, but if you focus on the most exceptional success, they can regress to the ME so much.
Then the second best, for example, if you analyze in an industry or context together. Then you can have this inverted U-shaped pattern. So higher performance indicate higher skill or higher future performance, but up to a certain point. Then you have this negative correlation for the top. So the most exceptional, like musician, I study most exceptional ferns.
I study. Their performers are not that persistent, just like the the 50 organization feature in this best seller. So they just when bankrupted underperforming, they cannot really build to less right. But no one checked that. That is the bias comes in. People just attracted to this exceptional success and wanted to learn from them so that it can be equally successful.
And forgetting that well, luck is not replicable by definition, so what’s most likely to happen is disappointment.
Matt: It’s a fascinating. Quite deep psychological need, isn’t it, to be able to spot winners. There was a few years ago the dunno how you’d describe him these days. Darren Brown has been a stage hypnotist, a magician, all sorts of things. He did a program which was somebody being offered a bet on horse races and over the course of the show she was given tips.
Which won and won over seven or eight races, and then it got to the eighth race and she was offered the chance to be able to place a bet. On this last race. But then at that point, if I remember it rightly, he then revealed how this had been achieved and the way it’d been achieved as he started.
I can’t remember hundreds of people who’d been offered bets at the first stage, and then somewhat less and somewhat less, which of course meant all the magic went out. Any good magic trick actually, when you find out how it’s done it, it makes it. Usually quite dull. But that, that for me, really stuck with us is this idea of the survivorship bias, the, just looking at the ones that are successful and then thinking that we can draw from those.
But it, it does seem to be quite a, um, a deep rooted, it’s more than almost a bias, it feels like it’s almost like a societal need. We have to be able to look at winners to find out what is their secret.
Chengwei: Yeah, absolutely. I think your statement is definitely true. It’s consistent with my experience in the sense that I don’t teach these ideas about luck and the implication to the MBA students, less and less, but I teach that to exactly MBAs or undergraduate. And the reason is the following because.
This survivorship bias of course it’s a bias, but all the success stories drawn from this bio sample, they can have this motivational effect, right? That’s why all these kids, we give them all this autobiography of the heroes and heroing in history that try to motivate them, that if you. Try your best.
You could aspire to these role models, but of course, according to my research program, I would suggest Napoleon, all these people, they have great achievement, but there’s a lot of randomness be beyond the effect, beyond their control, foresight, also contributing to their great achievement, right?
But this autobiography or whatever did not really talk that much about that. So that’s a misleading account to the kids. But it’s also have a evolutionary functional, adaptive function. Because if you tell the other story yeah, all this a hero, they’re just lucky. And then the kids will lose all the motivation, right?
Why not? They just don’t study at all. Just waiting for the social security to save them. Then our society collapse. So I think yes, there’s a very viable account to believe that. All the survivorship bias. They stay in the human cognition for adaptive reason. However, in the past 300 years, the evolution of technology, the pace is so fast, it’s outpaced how our cognition can evolve, right?
So there’s a lot of mismatch. All this billionaire, and now we’re talking about trillionaires. There’s a lot of factor about luck playing their success and learning from them. Just going to lead to disappointment no matter how much you can copy their practices. You can’t copy their log. So that’s become a more detrimental way to learn them.
Then back to my teaching practice. The full-time ba that is an interesting group of students. They’re excellent, actually because they all many of them, I know we’re in top investment banker investment banking firms or top consultancy firms, and that’s what allow them to enter top MBA program, a master business administration program.
But they also suggest that they are at the point of their career that their confidence is probably higher on average to their competence. So this is a source of confident overconfidence, right? They really believe they have the full control of their destiny. They can achieve everything. So this idea about the more successful you are the less you should attribute your success to to yourself, but to other things, they don’t feel comfortable about that.
And that reflected in my teaching ratings. So when I try very hard to convince them with evidence models, empirical data. They push back even more. Probably not in the class because I pretty confident my evidence is convincing. But of course, teaching school got punished because of that.
However, for the exactly, MBA student usually 15 years or older than this full-time B student, usually EMBA are being there late forties or even fifties. They have seen enough in their career people can have this lucky success. They don’t deserve or very good people, they can suffer from some bad luck and never recover from that.
They sing enough and appreciate. Yeah, this is a probably very realistic account and more constructive if you really want to learn properly from your experience and others lessons. Yeah, I teach this idea in my even EMBA and exec education. Usually they’re much more better received than the younger, but equally excellent student.
But they’re just in their different time of their career, they cannot really absorb this idea in a constructive way.
Nick: Does that make so I’ve been listening intently, scribbling down questions and scribbling them out as you’ve answered them. Like one of my one of my questions was gonna be how many people hate you? ’cause this is I think Matt made a really good point about how what you’re saying is a fundamental challenge.
To what people believe, and I would go as far as to call it a dangerous idea in that so much of your success in the future or the success of people who maybe you look up to or you want to imitate is down to luck, always down to things like be in San Francisco 20 years ago is the, is like the big tip and you can’t do that.
But is that self-belief? Like table stakes to then being able to be lucky and even if one of your more mature students pays attention to what you’ve said, that they need to ignore what you are saying about luck and over believe the influence they have in order to open themselves up to luck.
Chengwei: Yeah, absolutely. Nick, I think this very profound questions you asked, right? Probably I can have two answers. The first one is that, yes I do believe if I want to be a good competent intellectual, I do need to have some enemies. And I really opposing my ideas Arch enemy. But interestingly I think we’re in a generation with huge social inequality.
So my account coincide the time that more and more people appreciate that. Probably not for the evidence reason. They just found this the wealthiest people. They’re not necessarily envious, but they sometimes just feel this richest people. They do not deserve all the wealth. And my research happened to support or provide some evidence.
Yes, that’s probably the case. So they may like my research program for kind of a slightly different reason, but I think my, my account. Probably will attract some eyebrow raising from the richest and most successful people, right? I’m not invited to sitting in any board of this large corporation.
I’m actually advised several startups. Not exceptionally successful yet but not all this large corporation, the big 500 for example. But that’s just anecdotal. Account. But I do think that we are in an unfortunate time that increasing equality in many of the developed. A developed country actually make this account actually popular or being talk about or slightly become more, more mainstream to some people because in my field, actually there’s a special issue in one of the top journal just talk about this role of luck in a very explicit way 20 years ago, no one.
Kind of talk about this. Actually, when I share my idea or my dissertation topic on lot people will say, wow, why do you destroy your career? You’re not going anywhere. Particularly the business school, right? Business school. We’re talking about capability. We’re talking about developing superior skill and decision making, and then you talk about the opposite.
Then of course, people got confused. And that, like I mentioned, the second book is about contrarian, right? I guess I’m a kind of a contrarian mindset from the very beginning when everyone in my PhD cohort study XI want to study Y. It turns out to be the topic of luck. And the second question, I think it’s actually a very consistent to what I told the EMB student or the senior student, because in my research program I actually say there’s a lot of conventional wisdom about luck.
I fully agree. Chances, favor the prepare mind. If you work harder, you get luckier. I fully agree with them as long as you’re talking about like a normal range of performance, right? If you teach a undergraduate student or a master level student in their twenties, of course there’s a lot of things we can teach them to avoid making more mistakes and pick up some of the good practices to improve their performances.
But. For the most extreme performance the top level performance, there’s no rule to become the richest from good to great. The role of being at the right place at the right time is much more important than skill and effort. However, that’s consistent with what NICU said. Like the EMBA student, they’re successful enough.
They’re probably already from poor to good. They’re good enough, but whether they can move from good to great. They need to be more humble and open up their network, for example. And all the serendipity research applies, right? So they cannot really have full control of their whether they’ll get lucky or not, but there some systematic where they can increase the chance of having that particularly there already in a position, fortunately not position in their career.
To connect it to the right people or have the right resources when the opportunity emerge. So I think to them the research finding I presented actually the resonate even more because of that.
Nick: With that situation is that just having very quickly looked at literally a paragraph summary of your ideas around being a smart contrarian, is that where that comes in? As in, if you’ve. Got lucky to get to a position, as you say, at a good level, to get to a great level. You can take advantage of that luck and actually be more intentional about how you use ideas, how you realize.
As much as it, as painful as it might be, that luck got you too good. And you need, you can rely on more thoughtful methods to get to. Great. Is that where your interest in being a smart contrarian came from?
Chengwei: Yeah, again, that’s an interesting question, so I’ll probably have two answers to that again first of course, yes, that’s right. If you are lucky enough to be successful enough, then this is the story about the rich guy richer, the poor got poorer, right? If you are already in that position, getting lucky actually become easier because like I said, if you’re luck, if you are successful enough, then good ideas come to you.
Good people are more likely to working for you or your organization. Everything become easier. That is just like a reinforcing mechanism, right? So yeah, so the already successful people, if they get luckier and have more opportunity open door to them, that is actually not very surprising, particularly in a highly unequal society.
Then all this opportunity concentrating on fewer people, right? If you think about the AI development, that’s the thing, right? Normal people cannot even invest in this company. But when it’s a. The big five, they just keep throwing billions and billions of money in this organization. Based on the trajectory, they’re just going to dominate all this wealth, even in a more concentrated way than what happened in the past 20 years.
Yeah. So unfortunately that’s the one aspect. Then the other is why I studied this smart contrarian actually that the initial story was my very d desperate move to my very low rating for the full-time ba because. When I teach this idea, I mentioned they didn’t like it, but I still need to teach this course.
And one of the thing I told them then is that, okay, so you have learned these ideas, but people outside this room, they haven’t this knowledge as symmetry. Now you can do something about it to translate the biases of others into your own advantage or profitability. So that is the origin. I how do I link to the lock research program to being a contrarian.
So let me give you a specific example. So how to win a Fair Lottery game. So whenever I ask this question to my MBA students they’ll come out with all sort of answer, and one of the very popular one is that why not buy all the tickets? But then I challenge them, what is the expected payoff?
Then they realize, remember the economics 1 0 1, the lesson, right? The expected payoff. Actually it’s zero if it’s a fair lottery. Of course there’s a implementation hurdle, right? You may not be able to really buy all the tickets and all the mistake, all the oversight may lead to huge losses. And also the organization probably has on charity purposes.
So the expected pay of actually is negative. So if the game you’re playing, the expected payoff is negative, you should not play, you should walk away. So that is, of course, that’s the decision making code. That is a very important lesson to be learned, right? Uh. Understand what is the game you’re playing?
Is that the right game? It’s the wrong game. You just walk away. You don’t engage. So the question I then ask them is that if you have to play a fail out your game, is there any strategy. To win or enhance your expected payoff, then usually they get puzzled. Then I gave them my hint. So if you have to play this fair lottery game, which we already clarify, this is the unfair one to you, but you have to play, always pick numbers greater than 31. And 31, of course, that many student then realize it’s probably related to birth date, right? So it turns out this is backed by data. So the Fair Lottery game in this case, take a typical lottery is one to 48. So the. Probability of each number being picked is equal, uniformly distributed, but the way people choose among this 48 number are equal.
The distribution, actually this asymmetrical, if you study the data, then one of the data studies from the Dutch data, how people choose the lottery number. Then one to 31, the bar are much higher than 32 to 48. The reason now is not too difficult to imagine, right? Because a lot of people believing that choosing the number, the birthday of their own or their relevant one, are more likely to get them luckier, more likely to be chosen.
But that is not the case because this is a fair lottery. And in most countries, lottery is indeed fair. It’s if you study the lottery number, uh, being chosen is indeed uniformly distributed. But then it suggests that arbitrage opportunity. Let me make it very clear. If you choose number greater than 31, you will not be more likely to win.
But if you’re lucky enough to win, you share the price with fewer others. That’s how can enhance the expected payoff. And that’s really capture the main idea of my new book. And of course, using the example, this is about the superstition about luck or this full by randomness, right? So if your rivals, competitors, people out there have some biases.
They don’t get it. They flow by randomness. Using their knowledge, you can translate their biases into your own advantage. Just like the lottery example, I’m not encouraging gambling, but I found this is a very quick example. Can help idea to understand how to be a smart contrarian, if to choose the right moment, and also backed by data and evidence about doing things differently can help you.
Matt: That’s really interesting. And of course there are so many of these biases that affect into questions around things like probability. Doing a short video recently about the gambler’s fallacy and the idea that the, it’s, it is the misrepresentation in our brains about how randomness works. If you’ve got something like a dye.
Being thrown that the last throw has no impact on the next throw. But our brains go, oh, but you’ve thrown three one. So a fourth one is just impossible, isn’t it? And tho those kind of biases alongside things like cultural factors like birthdays is yeah. Really interesting from all of this work.
When we come back to the idea of being able to make. Decisions on a day-to-day basis in a working environment, or I guess even outside of work, but particularly in a working environment. Are there other sort of generalized ideas that you have to be able to help people acknowledge the randomness factors and the luck factors that are at play within how things actually pan out?
Chengwei: , I think sometimes to improve decision making had this decision flow chart and checklist actually that’s a very effective way to eliminate this randomness, right?
Because there’s a lot of evidence, as we saw in my research, also showed that. We can, our decision making is really inconsistent. It’s not because our decision are inherently biased, just because we are human beings. A lot of situational factors irrelevant to our focal decision can influence our input in in a almost a neural level.
And we don’t recognize that. So that is the randomness comes in your decision making, polluted your decision making process, and the simplest way to, to avoid that is to have a checklist or have a decision flow chart. And you just follow that decision flow chart whenever you need to make this important decision.
Or a checklist, like a, for the doctor, they have the checklist for the procedure. That’s not for the doctor. Only for the nurses. For all the staff involved, or for the pilot before they take off or land. Because if you don’t have that. Then you can make this a small error and they can cascade in a complex system lead to major disaster.
But if you have that and follow this default decision flow chart or checklist very carefully, then the outcome can have a kind of baseline performance. So I think that would be one of the way one of the approach. I highly recommend if you are in a domain that like the randomness deviation in general is.
Bad, then I think it’s really useful to have that. But it’s really context dependent, right? Because some of the fields like creativity. Creative worker or professional actually deviation can be considered positive. So you don’t want to have all this idea that just like replicating the past, right?
You want to deviate from that in a more constructive way. Randomness, deviation, and some people call serendipity because of course serendipity is again, survivorship bias, right? It’s randomness, but having to be associated with some positive outcome. You want that in some domain.
Like a musician or a creative industry. So then you don’t want to have this decision flow chart or checklist can eliminate all this randomness, right? So you actually want to create encourage people to think more wildly. I think your book then the structure and the design capture that, right?
You have the random flow or the combination, all these different cards. Then people are more likely to come out with their own way of interpretation and they’re usually very good. So that’s how I think about that. And sometimes in another device I wrote about is that we should allow randomness to actively play in our important decision making as well.
So I recommend to my recruitment committee that after short shortlisting the candidates for like a faculty. Like a three to five, right? We probably short I mean there’s a lot of candidate, but after a three rounds of shortlist, we probably have three candidates. In the last round, and I recommend, oh, in that stage, we should probably just use deciding which to hire by lottery rowing, a dye, or just like a spreadsheet, random number generator, right?
Who to hire then of call. The idea was a veto. Even they see the reason because in many organization we do believe this rationalization is so important. We need to deliberate. We need to find why one candidate is much better than the other two. Or not much better than the other two. And we can justify that decision.
But what I observe in a lot of research, in organization behavior supported that well, when the differences among the shortlist candidates or if you think about technology choices the final set of the technology when the difference is very small. Then you still try to deliberate which one is better.
What happened is that there’s more pathway to introduce bias in this decision. The unconscious bias or whatever, or politics comes in. And randomness, of course, cannot guarantee you find the best possible candidate among three or the best technology among all the short listed one, right? But they can sanitize a lot of bias in politics also.
Or more, obviously save a lot of time. You don’t need to join all this meeting and people just talk and talk and actually do not contribute anything meaningful to the final decision. So I really think randomization. We should copy the Greeks, ancient Greeks and ancient Italians and to use this randomization in modern society, more particularly at the final stage of the decision making.
Wham. Choice said the differences among them are very small. Then deciding by lottery actually can lead to better outcome in a more efficient way.
Nick: it’s been like 40 minutes, so I like to think we know each other well, so I’ll say this slowly as best as I can. Where does your confidence to take on something that’s so essential to the way people think? Where does that confidence come from? Because what you’ve just said is when the decision feels the most important, when you’re down to the final three and the final three candidates, and it’s such an important position, you’ve got the best of the best.
So what we’re going to do is rather than say an interview table of seven important people whose time is really valuable, and they’ll wanna say something they’ll wanna ask their pet question is, save all that time, roll a dice. It’s that one, three to four. It’s that one. Five to six. It’s that one is, you are absolutely right, but I can almost feel the force of human nature that you are.
Up against, and I’m genuinely impressed and I’ve got some kind of, king canoe mental image of that. Never watched it. Game of Thrones of the single guy pulls out his sword as you’ve literally got a screen of calvary towards him and you’re like, no, this is right. Just roll a dice.
Pick one. Let’s take six months to find out who’s the best. Where does that confidence come from in your manner or background? And how much effort do you have to spend to keep it going?
Chengwei: Yeah. I wouldn’t say I’m very confident in implementing that. I’m confident in that REN selection can lead to equivalent outcome. To deliberation because I can provide mathematical proof because the model is very simple. If you have the candidate pools and just apply the same criteria that the higher performer, they stay in the system basically promoted in and out kind of system, right then you can prove at the end of the selection when you have a few amount, all the candidate consider the difference is amount them, is timing.
Whenever that condition is fulfilled, then ance selection, actually they have no downside of doing that. But then all the decision making approach can have a lot of downside relative to ance selection, particularly when you introduce the po potential of the Hama. So if the research committee or the selection committee, you can randomly model several background and then more likely to choose the people more similar to them.
That it actually creates some downside, right? And then read selection won’t have that. So that’s almost a mathematical proof that under many normally occurring situation, incorporation in selection, read selection actually can be better off because they avoid this predictable downside of a homophily and political.
Aspect, but I’m very underconfident in implementing that. To some extent, I plan the enlightenment because why the ancient Greek can do that because they don’t see, they don’t follow any rationale. I mentioned when they use election, they actually. Friendly, more a divine will, right? So that is chosen by the God and as God will and everyone will accept it.
But after enlightenment, this rationalization the rational decision making. Take a very central role. You need to justify everything you decided. Reason and randomness is opposite to that. There’s no reason. And that is so incompatible with the corporate rationale or decision making in modern world.
That’s why in the internal implementation there’s so much pushback. People feel uncomfortable. So some of the suggestion I did to this, almost like playful, is that for the exact education, they’re the CEO or the board member, right? So I said to them, if you do believe what I said. Implement that in private, do not share the idea or the practice that, okay the chosen executive actually said, selected by lottery.
Then no one will listen to this leader. Or a lot of people will have a lot of doubts, right? So why this guy or lady was chosen? Just not any justification. Just because they’re more they’re luckier than others. That is not defendable when enough people still have this bias in favor of rationalization.
So yeah, I’m confident in the intellectual foundation why this is. Good enough for a lot of decisions, but also in the social aspect or behavior aspect of implementing that in practice. I’m not so confident, but speak of that today. I’m not preparing for research grant proposal and really pleased to learn that what happened in the Switzerland now happening in the UK as well.
There’s a British Academy a grant application. They say very explicitly. Following, almost like my recommendation. They will evaluate each proposal with on the meritocracy, but then for the second stage, for all the proposal that passed that stage, they’ll do the redness selection. So they’ll only give money to the lucky one.
And of course they’ll tell everyone this is what happened. But interestingly, in Switzerland, they already demonstrated because they implement that 10, 15 years ago, that they track the productivity of this lucky team and unlucky things. Basically, there’s no difference. So for the proposal for the research team, they already produced good enough proposals.
Then their long-term output is equivalent of course one of the conditions that those unlucky, one, they need to keep trying, right? They need to apply for other grant also, like render selection. If you have large enough sample size, eventually it doesn’t matter. So yeah, that’s another supporting evidence that well, yes, random selection can be useful for modern society.
At least save a lot of time for this very long and unproductive meetings.
Nick: It’s that aspect, it’s the saving time plus. If you didn’t get through, it’s not that you were worse or that the interviewer or the person reading your application didn’t like you, or you weren’t the same background as them by chance, and it’s that you were unlucky, except luck isn’t really a thing that attaches itself to people.
So you’re not unlucky. It’s just random. Try again with a different grant funder or whatever. Feels like such an obviously. Good idea. And also I’m sat here looking at both of you. I dunno if Matt wants to include this in the recording of just like the, some kind of secret company that comes up with justifications for just randomly picking things and okay, it seems like your company will use, we’ll use something religious. We’ll use serendipity that fits in with your culture, and it’s chosen randomly, but it’s god’s will that this person’s the CEO for someone else. We’ll just take them on holiday for a while and really, but really we’ll just pick one and it’ll be half as much as your people actually interviewing them.
It feels good way of saving so much resource, and it feels relatively obvious once you think about the system, but also I can imagine having this conversation like 20 years from now lamenting that the situation is still the same as it is
Matt: one could argue that we have these companies already and they’re called things like Garner.
Nick: Wow, that’s a, Chang Ray’s Next grant application should be for studying something like that, which will generate him even better enemies than got so far.
Chengwei: yes. I love that. And there can be, company being said, the governor is the increasing example. But I do have a caveat for that because some of my PhD student actually wrote a proposal, theoretically haven’t tested, but it’s really believable if we take like the gender and racial bias. In that because me, looking at this grant proposal and this randomness involved, I don’t care.
I just finish that and submit. But there’s a lot of research suggest that like a female, they’re more likely to be terd by this type of situation or after they’re being selected or not selected. Even the the mechanism is not. Within there it is not meritocracy or in this case, luck, right?
They’re less likely to try again because they’re more likely to attribute this failure to themselves. Even you tell them the mechanism is bad luck. So yes, this can create some unfortunate kind of a gender imbalance or kind of large and large this gender imbalance. The proposal I gave, of course it’s defendable from certain perspective, but a lot of the new ones, what is a long-term impact for different people, different demographic profile that is still unknown.
So that is a nuance to, to this. But the idea about setting out a firm just like a, calling this random device to. Like we have the latest LM model to make the best decision for you, whereas it’s really just a lottery at home that may fool a lot of people. Yes, exactly.
Nick: Oh, I should stress to the listeners at home or jogging or whatever. I just held up a DA single dice, which yeah that, that’s a sort of backend process to selecting the best. But that’s a great point.
The cultural connotations of that are huge and would vary a lot from. People like us, especially something I hadn’t thought of, especially by gender, is feeling that it’s somehow attached to you when it will literally be a random number coming out of a hopefully random computer. That means someone didn’t get a position or didn’t get a grant.
Speaker 25: Yeah, just one more thought on that from Taiwan, right? The cultural differences. You mentioned Nick, that play a huge role, right? Because luck is not neutral. In, in, in some culture in Taiwan, people will consider, again, destiny fate. So if you’re not chosen, even we know it’s just like a dice, a random device, right?
But a lot of people will interpret that very differently. That’s why the long-term consequential readiness selection, actually there’s a lot of things we don’t know yet.
Matt: A fantastic conversation. Thank you so much for joining us this week, Chengwei, um, although I do feel that the three of us might have quite a lot of kind of common vested interest in these subjects. I don’t know, um, what have you got, uh, coming up in the week ahead?
Chengwei: Yeah, I’m, uh, very excited. Uh, I’ll be in the us I’m invited to give two talks, one in MIT. Uh, and the other in the Dartmouth, uh, college Boston. In Boston. So I really look forward to, um, the, this visit and meet, uh, some of my old friend there and also, um, wow. Uh, very sharp and critical researcher. They’re known for that.
They, they try to destroy you in the seminars, but I look forward to the challenge.
Matt: That sounds slightly scary, but I hope you have a fantastic time. Um, Nick, how about you? What have you got for the, uh, the next seven days?
Nick: I think. Try as, um, cliche. Does it sound trying to be lucky? Like just, I was just looking at my schedule and just having various online meetings with people and trying to increase my luck surface area, which is another. I think we sort of, I think, skirted around but didn’t get into it. That is, um, just talking to people who, who I like and get on with, but, or who also might be useful and thinking about how to make that happen is my next week.
How about you, Matt?
Matt: I will be going for a farewell dinner tomorrow night for one of my colleagues who’s going to be leaving. Uh. Equal experts in a couple of months time, but because summer and stuff we we’re having a, a farewell dinner quite soon. Uh, other than that, um, and other than tracking the ongoing process progress, sorry, of the OOCL.
Peu, uh, which is the, the boat on which the book is sitting. I think I might have become a bit obsessed. Uh, I am, um, I’ve actually got like just bits. It’s one of those weeks ahead where it’s just bits, but then we’ve got a long weekend and so probably the long weekend will mostly be about geeing up eldest as he gets ready for.
The, um, the sheer madness of GCSE exams. He had his first one today, but it really starts in earnest in about three weeks time. And so getting him ready for that. So that’d be fun. That’d be the first of four years back to back of doing school exams by proxy. I can’t wait. I enjoyed them so much when I did them.
Um, so with that, I think we’re about at time. Um, Chengwei once again, thank you so much for joining us on the show this week.
Thank you.
And we will be back at some point after the, the bank holiday break. It will either be the week after or the week after that. I’m trying to line up a guest at the moment. And so, uh, we will continue sporadic and somewhat random timing of output because that’s the way we like it here on WB
40.
Chengwei: Thank you for listening to WB 40. You can find us on the internet@wfortypodcast.com and on all good podcasting platforms.
Leave a Reply