Chuck Darwin<p>More from Dan Davies:</p><p>About five years ago, I started to get very interested in an obscure subject called “<a href="https://c.im/tags/management" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>management</span></a> <a href="https://c.im/tags/cybernetics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>cybernetics</span></a>”. </p><p>It was a product of the technological dreamscape of the 1960s and 70s; </p><p>after the invention of the computer, but before it became ubiquitous, <br>in a period when there was room for speculation about how the new world of artificial intelligence would change our world.</p><p>I had just finished my previous book<br> (Lying for Money: How Legendary Frauds Reveal the Workings of the World), <br>and was keen to say a bit more about how organisations go wrong. </p><p>It seemed to me that it might be possible to expand the concept of a “<a href="https://c.im/tags/criminogenic" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>criminogenic</span></a> <a href="https://c.im/tags/organisation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>organisation</span></a>” <br>(one where the incentives structurally produce illegal behaviour) <br>to a more general 🔸“bad-decision-o-genic organisation”.</p><p>And furthermore, that the weird mixture of pure mathematics, philosophy, accountancy, physics and economics that came together in the work of now-forgotten management gurus like <a href="https://c.im/tags/Stafford" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Stafford</span></a> <a href="https://c.im/tags/Beer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Beer</span></a> and <a href="https://c.im/tags/Barry" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Barry</span></a> <a href="https://c.im/tags/Clemson" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Clemson</span></a> might be the way to think about it.</p><p>What do bad decision-making organizations have in common❓</p><p>Quite a few things, <br>but one of the clearest signs is something you might call an “<a href="https://c.im/tags/accountability" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>accountability</span></a> <a href="https://c.im/tags/sink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>sink</span></a>”. </p><p>This is something that might be familiar to anyone who has been bumped from an overbooked flight. </p><p>There is no point getting angry at the gate attendant; <br>they are just implementing a corporate policy which they have no power to change. </p><p>But nor can you complain to the person who made the decision <br>– that is also forbidden by the policy. </p><p>The airline has created an arrangement whereby the gate attendant speaks to you with the voice of an amorphous algorithm<br>-- but you have to speak back as if to a human being like yourself. </p><p>The communication between the decision-maker and the decided-upon has been broken <br>– they have created a handy <a href="https://c.im/tags/sink" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>sink</span></a> into which <a href="https://c.im/tags/negative" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>negative</span></a> <a href="https://c.im/tags/feedback" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>feedback</span></a> can be poured without any danger of it affecting anything.</p><p>⚠️This breaking of the feedback links is, I think, one of the most important things that has happened to large organisations <br>– banks, but also large corporations and government departments<br> – over the last fifty years. </p><p>In most cases, it’s not been carried out purely as a responsibility-dodging exercise, <br>or as part of a conscious effort to make things worse. </p><p>That has happened, on occasion, <br>but for the most part, after spending a lot of time looking into examples, <br>I concluded that feedback links were being broken simply because 💥they had to be. </p><p>The world keeps growing and getting more complicated, <br>which means that individual managers gradually become overwhelmed; </p><p>the problem of trying to get a sensible drink from the firehose of information that pours into any large organisation every day has become unbearable.</p><p>And this is why institutions have started ❌ delegating decisions to systems <br>– credit scoring algorithms, regulatory risk weighting formulas <br>and the like. </p><p>As well as allowing decision-making to be automated and industrialised, <br>they provide a psychological defense system, <br>👉preventing individual human beings from the consequences of having to make a decision and own it.</p><p>Most of the time, these systems work well. </p><p>But when they break down, the consequences can be spectacular. </p><p>Because every such algorithm or rulebook is, implicitly, based on a <a href="https://c.im/tags/model" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>model</span></a> of the thing they’re meant to govern. </p><p>And every such model is capable of failure. </p><p>And when something comes along that’s outside the model <br>– like, for example, a sustained nationwide fall in US house prices <br>– you end up in a situation where literally nobody knows what to do.</p><p>(1/3)<br><a href="https://www.netinterest.co/p/decisions-nobody-made" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">netinterest.co/p/decisions-nob</span><span class="invisible">ody-made</span></a></p><p><span class="h-card" translate="no"><a href="https://mamot.fr/@pluralistic" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>pluralistic</span></a></span> <br> <a href="https://c.im/tags/Boeing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Boeing</span></a> <a href="https://c.im/tags/737MAX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>737MAX</span></a> <a href="https://c.im/tags/Boeing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Boeing</span></a> <a href="https://c.im/tags/merger" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>merger</span></a> <a href="https://c.im/tags/McDonnell" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>McDonnell</span></a> <a href="https://c.im/tags/Douglas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Douglas</span></a> <a href="https://c.im/tags/engineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>engineering</span></a> <a href="https://c.im/tags/culture" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>culture</span></a> <a href="https://c.im/tags/cost" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>cost</span></a> <a href="https://c.im/tags/control" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>control</span></a> <a href="https://c.im/tags/Ricardian" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Ricardian</span></a> <a href="https://c.im/tags/Fallacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Fallacy</span></a> <a href="https://c.im/tags/hard" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>hard</span></a> <a href="https://c.im/tags/data" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>data</span></a> <a href="https://c.im/tags/culture" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>culture</span></a> <a href="https://c.im/tags/best" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>best</span></a> <a href="https://c.im/tags/practice" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>practice</span></a></p>