In pt. 1 we concluded that in order to comprehend reality, we must free ourselves from ideology.
With minds unshackled, we will now focus on the three fundamental tools required to accurately assess the most pressing prediction of any given day;
Will dinner be good?
1. The Wisdom Of The Crowds
One of the greatest boons to forecasting involved a local market, Charles Darwin’s cousin and a question over beef.
In 1906, Francis Galton wanted to test a hypothesis that people were at large stupid. He found his perfect experiment in a competition at a local fair to guess the “dressed” weight of a whole ox, where “dressed” is the slightly macabre term for the meat products once butchered. He analysed the 800 guesses the townsfolk made across the day and noted the median value of 547kg. He expected this to be barbarously inaccurate, which would then prove his point about the common man’s ignorance, and therefore provide more fuel for his future career as a leader of the eugenics movement (Galton wasn’t quite as nice a bloke as Darwin).
However it turned out people are not so stupid, and that aggregate views can be uncannily accurate. The dressed ox weighed 543kg, meaning the median guess was only 0.7% off the exact weight. This was The Wisdom Of The Crowds in action, a theory that declares certain decisions are made better by aggregating a diverse collection of individuals’ views rather than by referring to sole experts.
The wisdom of the crowds is at our fingertips in the digital era. Prediction markets provide odds on questions from whether humans will land on Mars by 2025 (the crowds give 96% no) to if the next 007 will be a woman (40% going for yes… the name’s Bond, Janis Bond).
When assessing what to cook, we can access the wisdom of crowds by typing an ingredient in Google and seeing the most common searches. I’ve put some examples in the image below, and would myself vouch for many of the suggestions (though I be but one man).
2. Definitely Yes. Definitely No. Maybe
Barack Obama was presented with the wisdom of the crowds when deciding to give the go ahead to Operation Neptune Spear, the mission that killed Osama bin Laden. This time he wasn’t shown the wisdom of an online prediction market, nor of a google search auto fill-in (“Osama Bin Laden is hiding…” ), but that of the CIA.
National security analysts submitted odds of Osama being in the suspected compound in Pakistan that ranged from 30 to 95 percent. The CIA director explained to Obama that the aggregate probability lay at about 70%, to which Obama reportedly said ok so “this is fifty-fifty.”1
Despite being confronted with one of the world’s most critically calculated forecasts, the great leader could not help but fall into the trap of declaring the quoted odds as equivalent to “a flip of the coin.” This - as Daniel Kahneman would have it in Thinking Fast, Thinking Slow - is because humans are prone to three types of conclusion when considering an outcome; definitely no, definitely yes and maybe.
The evolutionary psychology (evo psych in the business) behind this naturally slips straight into the arena of hunter-gatherers and lions. If you are wondering if you saw a lion, you will think either:
Definitely no, in which case you carry on with business.
Definitely yes, in which case you prepare for whatever you must do when there is a lion (a frightening dance?)
Definitely maybe, in which case you prepare for whatever you must do when there is a lion (carry on with the scary dance).
30% versus 90% probability of seeing a lion is not important when calculating how to respond. Any odd is enough to encourage a reaction, and the idea is we are therefore primed to immediately treat all degrees of maybe as the same.
The key here is immediately. This is the System 1 mind that is instinctive and acts without leaving room for costly mental flip-flopping. What humans have been blessed with on top is the System 2 mind, which when accessed with time, can more critically analyse the probabilities.
Super-forecasters are counterintuitive System 2 thinkers. They understand probabilities are graded and adjust predictions by increments as small as a single percentage in light of developments. Predicting if dinner will be good is not a question of definitely yes, definitely no, or maybe, but will be a summed weight of the contributing factors. Fresh ingredients: plus 5%, not following a recipe: down 25%, eating the same thing you ate yesterday and know you didn’t even like then: ok, well, minus 99%.
3. Fermi-izing
When he wasn’t building the atomic bomb, the nuclear physicist Enrico Fermi enjoyed asking his friends questions like “how many piano tuners are there in Chicago?”
This was not because he lived in Chicago and had a piano and it needed tuning. Instead he liked to see what you could know about what you don’t know by using what you do know. His technique was to take an unknown like the piano tuners in Chicago and unpack it into sub-questions:
How many pianos are there in Chicago?
How often are pianos tuned each year?
How long does it take to tune a piano?
How many hours does the average piano tuner work?
Some of these questions are almost as unanswerable as the first, so we can break down further until we get to something we do know. We can ask how many people are in Chicago? Well it’s quite a big American city, so let’s go without about 5 million. How many pianos are there probably per 100 people? I don’t know, 1 seems reasonable?
I continued with these kind of estimates and ended up with a guess of 100 piano tuners*. Researchers bent on actually answering Fermi’s old riddle found 83 listings for piano tuners in Chicago. With only a few extra nuggets of information (how many people actually live in Chicago, how long it takes to tune a piano), we would have got even closer.
If I was to ask you how likely a terrorist attack would be in the UK between July and October, what would you do? My initial reaction would be to read the news, check various Facebook groups for brewing unrest, hack the MI5 database using my coding skills, and so on. However the Fermi approach would be to say, well how many terrorist attacks are there a year in the UK? If it’s 1, then the odds of one happening in that three month period are roughly 25%.
This is the starting view upon which we can then use the news (and Facebook and the MI5 database) to make small adjustments. Likewise if you were to ask “will your dinner be good”, before you even look in the fridge, the first place to go would be “how many times when I make dinner is it actually any good?”
Calibration Station
Above are some of the basic tools at our disposal for prognosticating, you may want to apply it to making sure your next dinner party is a hit, though equally you may apply it to more important questions in your life (if they exist).
However it is all futile if you don’t keep track of how you scored. The final piece of the puzzle is calibration. If you make a few predictions of 90% and each turns out incorrect, then you need to assess which weightings were flawed. There is no true progress without a feedback loop.
Perhaps you overestimated your ability to roll fresh pasta, maybe you didn’t factor in externalities like a crying baby making you burn the onions, or you put too much faith in a trendy cookbook with nice pictures that doesn’t in reality even have a scratch on Sam’s Good Stuff.
I’ll leave you all with an applied example from my own life. Below is the recipe for what I had for dinner this evening after finally getting a baby down to bed at 10pm and with hands stiff and tired from hammering at these keys in between stints of failing to put him down (to sleep, not to death). Included in it are my own estimates for its success and finally a calibration after the results came in.
*values of my guesses for piano tuning: 50,000 pianos in Chicago, 4 hours to tune a piano, one piano worker works 40 hours a week for 50 weeks a year so 2000 hours total. (50,000 pianos x 4 hours work) / 2000 piano-tuner-work-hours = 100 piano tuners .
Instant TanTanMen
These are indeed instant noodles, but everyone should know how to turn a pack of instant noodles into a legitimate (-ish) dinner. The process involves the addition of rogue elements lying about in the fridge, plus following principles of stock enhancement from the rabbit-hole of ramen subculture.
Forecast
Baseline: How often do I enjoy instant noodles? 60% of the time. Sometimes the slutty-ness fits, sometimes it doesn’t.
Reward vs input boost: A dinner’s goodness is definitely accentuated when the input was low. Noodles are instant and I already have some boiled eggs in the fridge (a good habit is to boil extra eggs whenever you boil them). 10% extra.
Joy of frugality and clearing up bits: There’s some scraps of rocket and watercress that I wouldn’t know what else to do with, and it’s nice when things just work out. 10% extra.
Using recipe from an authoritative source: Tantanmen is a Japanese ramen based on dan dan noodles, which as long time readers will know, I enjoy often. The ramen base recipe I took from The Book Of Ramen (free ebook), which is the most complete guide available on the subject. +30%
Could be too spicy: The ramen base plus the instant noodles spicy stock powder could be too hot. Though I like chilli, so not extremely worried. -20%
External factors: The baby could wake up and I may end up eating cold soggy ramen, about a 1 in 5 chance based on usual evenings. -20%
Sum: 60% chance I will enjoy dinner. This is much lower than I would have expected if I hadn’t forecasted and has slightly depressed me.
Ingredients
Instant noodles
Chilli oil (homemade ideally - mine is actually made with rendered beef fat using recipe in ramen ebook, plus 10%)
Tahini
Soy sauce
Eggs, random leaves, spring onions, sesame seeds, dried seaweed, bamboo, whatever you have.
Recipe
Make your ramen base by combining a heaped tablespoon of chilli oil and soy sauce with a table spoon of tahini, and combine well.
Cook the noodles with the stock together in about 300ml boiling water. Drain the cooking liquid into your ramen bowl and mix well with the base. Add noodles and toppings.
Calibration
The result was a hit. Does this then make my forecast of 60% inaccurate? For anyone who’s been listening carefully, it’s too early to tell.
Events are uncertain, George could have woken up, I might not have felt like instant noodles that day or it could have been too spicy. To properly test the odds, I would have to make such a prediction 10 times and see that roughly 6 of the times it’s good and 4 times it’s not.
Sound fun? I’m sure you and me will both be diligently calculating and calibrating before even the most routine meals now, all in search of foreseeing that most important demand of the day “will dinner be good?”
This story is told in The Finish: The Killing Of Osama Bin Laden by Mark Bowden
There’s a great dramatisation of the Bin Laden/probability discussion in “Zero Dark Thirty”. The organisationally savvy CIA bureaucrats circle around career-protecting both-sides-of-the-fence probabilities - “a soft 60%”. Maya, the heroine and source of the analysis that Bin Laden was in the compound in Abbotabad, is finally asked her view after the chaps have pronounced: “It’s a 100% … OK, I know you guys don’t do certainty, so 95% … but it’s a 100%”.
https://youtu.be/irG0wGzw7Ok
Outstanding. Feels like a first course in Empirical Philosophy.
First we put down ideologies, then we embrace uncertainty in order to fathom our next steps in the world as it really is…A tall order and often impractical (hence System 1) but you show us the rewards (depth of understanding) and remind us that writing past things down (calibration) plays an important role in perceiving what we write down next.
Tempting (for me) to add Feynman diagrams. The world at its root is deeply probabilistic, these little sketches show a way through. How to know what will happen in an interaction at the smallest scale? Identify every possible way the particles could interact, determine the probability of each, compile. Sum over all possible histories because all those things could and probably did happen. Everything everywhere all at once. At the macro level this is less a practical tool, more a humbling awareness.
How appropriate that your mediation indicated what the East can show us with a stop on the Way of the Ramen.