Process vs. Outcome

As I alluded to on the Interests page, under chess and poker, what I hold myself to is the quality of the process rather than the outcome. That started as a note about card games and turned out to be about everything else, so it gets a post of its own.

The short version is that outcomes are evidence, not verdicts, and that the only parts of any game genuinely mine to control are the decisions. The long version needs a basketball court.

A hundred jump shots. Give two people the same instruction: go and take a hundred jump shots. The first person hears a hundred repetitions of a movement, and their attention goes to the movement: where the feet are set, how high the jump is, whether the elbow stays under the ball, whether the release is the same on the ninetieth shot as on the first. The number is the container; what is inside it is the form. The second person hears a hundred, and their attention goes to the count: get to a hundred, by whichever means the ball goes through the hoop, or does not, and be done. Same court, same ball, same instruction; two entirely different afternoons.

Neither of them is wrong, and that is what makes the difference interesting rather than moral. The second person will finish first. The first person will be better at the hundred-and-first shot. If the goal is a number today, be the second; if the goal is a shooter, be the first. However, most goals worth having are the second kind dressed up as the first: a paper count is a proxy for a research programme; a year’s returns are a proxy for a strategy; a hundred made shots are a proxy for a jump shot. Aim at the proxy directly and you can hit it while missing the thing it stood for.

Two dispositions, each with a bill attached: Process vs. Outcome.

The process-focused person talks in terms of the journey towards something; of progression, i.e., getting a little better every day; of doing the steps in order and doing each one correctly before moving to the next. The upside is large and, in my experience, underrated: given the same amount of time, they learn far more than their outcome-focused counterpart, because a mind organised around how things are done transfers between different domains almost for free (the so-called cross-disciplinary thinking, or learning). The form of a jump shot and the form of a proof are not the same, but the habit of attending to form is, and so the process person connects dots across fields that look unrelated from the outside (this website is, in a sense, a demonstration of that). The downside is the rabbit hole: it is entirely possible to become so absorbed in the steps that the outcome you set out for in the first place quietly drops off the table, and you surface months later, much improved, having not done the thing.

The outcome-focused person talks in terms of control of the result, and of whichever method or route gets there. The upside is that they get the job done, usually, and on time, which is not nothing; most of the world runs on people like this, and a team with none of them on it tends to produce excellent plans, thorough notes and a prototype that is always one more iteration away from finished; somebody has to be the one who says the current version ships, and it is usually them. The downside is that the learning can be thin: the job is done, and sometimes they cannot say how it was done, or which of the things they tried was the one that worked, so the next job starts closer to zero than it should.

In essence, one disposition compounds knowledge and risks never cashing it in; the other cashes in reliably and risks never compounding.

It shapes the person. What struck me is that this is not merely a habit of attention; it hardens into a disposition. Do the hundred with your eyes on the form for long enough and you become someone who sees form everywhere, i.e., someone who, handed any task, first asks how it is done well. Do it with your eyes on the count and you become someone who first asks what done looks like, and works backwards from there. The personality literature noticed this long before I did, and I am a big fan of the people who did the noticing. In Linda Berens’ Interaction Styles, one of the axes along which the four styles are sorted is precisely a focus on outcome against a focus on process: two of the styles are organised around getting the result or controlling its quality, and the other two around having a process or being inside one. That is the basketball court, written down by someone who had watched a great many more people than I have, and it gave me a vocabulary for something I had only felt. What excites me even more is the effort to take the field from description into measurement: Dario Nardi, then at UCLA, spent years putting EEG caps on students of different types and looking for brain-activity patterns that track the typology, which is exactly the right instinct, i.e., stop arguing and go and measure. Personality typology is still a young science in that sense, closer to its first experiments than to its textbook, and I find that a reason for interest rather than dismissal: a field at that stage is where the interesting questions are still open, and where a process-focused person feels most at home, because the procedure is being built in front of you. I would very much like Nardi’s programme to succeed, and I expect that whatever survives replication will look a lot like what he found. Whether the habit builds the person or the person chooses the habit is, I suspect, a loop rather than an arrow; either way, I know which side of that axis I sit on, and I knew it from the basketball court before I knew it from any framework.

Why it governs how I read a number. Poker is where I learned this properly, because poker punishes the outcome view within an evening. You can play a hand perfectly and lose it; you can play it badly and win. Judge yourself by the outcome and you will learn the wrong lessons quickly and with great confidence. Judge yourself by the process, i.e., by whether the decision was correct given what could be known when it was made, and the outcomes start to mean something, but only in aggregate: one hand is noise, a thousand are data. Research is the same game with a slower clock. A p-value is one hand. A backtest is one hand. The question a process-focused person asks of either is not “did it come out the way I wanted?” but “would this procedure, repeated, keep producing it?” That is the whole of what a calibrated result is, and it is also, not by coincidence, the only question that survives someone checking your work.

A firm named after the idea. Finance worked this out at scale before I did. In 1993, Pete Muller pitched Morgan Stanley on a group that would trade on quantitative models rather than on traders’ judgement, and he named it Process Driven Trading: PDT. The name is the whole philosophy. What the group built was not a forecast but a procedure: form a hypothesis about the market, test it on data it was not fitted to, size the position by the uncertainty, execute the same way every time, and judge the desk by whether the procedure was followed rather than by any single month’s profit, because a single month is one hand. Nineteen years inside the bank, then spun out in 2013 as PDT Partners, with some two hundred scientists and engineers in New York and London today. It was one of the groups that made systematic, model-driven trading credible inside a major bank, and the quantitative industry that grew up in the decades after inherited its one premise, whatever else it changed: you cannot control the outcome of a trade; you can control the process that generates trades, and only the process compounds. My own quantitative work is built on exactly that premise, and it is why I treat a backtest as a statement about a procedure rather than about a return.

And in a neutrino detector. The same mindset is, quite literally, written into how DUNE and NOvA are run. An oscillation result is the outcome, and nobody on the experiment is allowed to aim at it directly. The far-detector data stay in a sealed box until the entire analysis (the selection, the systematic model, the fitters, the sensitivity studies on simulated data) is frozen and reviewed; only then is the box opened, and by then it is too late to steer the answer. NOvA runs one frequentist and two independent Bayesian fits and requires all three to agree before anything is published. DUNE, which has no beam yet, spends its years producing sensitivity studies at assumed true values, i.e., rehearsing the process before there is any outcome to be tempted by. A blind analysis is the process-focused disposition turned into a rule, and the reason it exists is the one poker teaches: a physicist who can see the answer while tuning the analysis will, with the best will in the world, tune towards it. My part in both experiments, the statistical modelling and the propagation of uncertainty from the near detector to the far one, is a part of that process and nothing else; when the box opens, the number will be whatever it is, and the only thing I will have controlled is whether the procedure that produced it was sound.

To sum up: form first, count second, and stay at the table long enough for the count to become evidence. That is the whole of what I alluded to on the Interests page, written out. I will keep coming back to it.