Feature Article: Thoughts for New Graduates: The Impact of System Factors, Skill, and Luck on Your Future

Like history, your future will emerge from the interaction of three causes: The nature of the system, the skill of human agents, and luck. In the years ahead, it will serve you well to keep some important points about each of these in mind.

The Nature of Many Real World Systems

If you’re like most students, during your years in school you probably heard a lot about systems dominated by linear cause/effect processes and negative (dampening) feedback loops that kept them in or close to equilibrium. You also probably used the familiar normal/Bell Curve/Gaussian distribution to statistically describe the results these systems produced.

Unfortunately, that is not how many large and very important real world systems work.

Instead, in complex systems many effects have multiple causes. A significant number of these cause/effect relationships are non-linear, and/or time delayed. In addition to negative feedback loops, complex systems also have strong positive (accelerating) feedback loops. Consequently these systems often operate far from equilibrium.

The results complex systems produce are often best described by power laws. Statistically, the variance of the distribution of these results is therefore not well defined. This means that it is very hard to use statistics to understand the range of possible outcomes complex systems can produce. The emergence of so-called “Black Swan” events is one of their common features.

Many complex systems are also “adaptive”, as they contain reasoning agents who adapt their strategies over time to achieve goals, which themselves may also evolve.

In Complex Systems, What Does It Mean to Be Skilled?

My favorite quote about the relative importance of systems and skills comes from Warren Buffett: “When a management [team] with a reputation for brilliance tackles a business [system] with a reputation for bad economics, it is the reputation of the business that remains intact." That often comes as a surprise to people who have been marinating in meritocracy throughout their years of schooling.

The truth is, when people are immersed in complex adaptive systems, most struggle to achieve their goals. In a recent paper, Anne Marie Grisogono (a complex adaptive systems expert who recently retired from Australia’s Defence Science and Technology Organization) summed up the research on why this happens:

“Low ambiguity tolerance was found to be a significant factor in precipitating the behavior of prematurely jumping to conclusions about the problem and what was to be done about it, when faced with situational uncertainty, ambiguity and pressure to achieve high-level goals. The chosen (usually ineffective) course of action was then defended and persevered with through a combination of confirmation bias, commitment bias, and loss aversion, in spite of available contradictory evidence.

"The unfolding disaster was compounded by a number of other reasoning shortcomings such as difficulties in steering processes with long latencies and in projecting cumulative and non-linear processes. Overall they had poor situation understanding, were likely to focus on symptoms rather than causal factors, were prone to a number of dysfunctional behavior patterns, and attributed their failures to external causes rather than learning from them and taking responsibility for the outcomes they produced.”

Grisogono also summed up what is different about those who manage to achieve their goals:

“They developed a [necessarily incomplete] conceptual model of the situation, and took actions based on causal factors, seeking to learn from unexpected outcomes. They constantly challenged their own thinking and views. Most importantly, they displayed a higher degree of ambiguity tolerance than the unsuccessful majority” (from Grisogono’s paper, “How Could Future AI Help Tackle Global Complex Problems?”).

Elsewhere, research conducted by Britten Coyne Partners, an affiliate of The Index Investor, has found that not considering time dynamics is another important source of failure in complex environments.

Here’s a simple example. In an investment context, consider an investor with $100,000 who wants to accumulate $133,000 after three years. Seeking high returns, they invest all their funds in the equity market, overlooking the fact that the variability of equity returns is higher than in other asset classes. In the first year their equity investment loses 20%, and is worth $80,000. To return to breakeven by the end of year two, it must earn a return of not 20%, but 25%. And 33% will be required in year three to achieve the investor’s goal.

When taking actions that have uncertain outcomes to achieve multi-period goals, significant setbacks reduce the probability of success by more than most people realize. That’s why smart risk management is critical in complex adaptive systems. By all means take risks – achieving ambitious goals is impossible if you don’t. But when taking those risks, try to find ways to limit your downside, such as creating robust plans (that will achieve your goals under a range of scenarios), building resilient processes (to cushion the blow when robustness fails), and cultivating an adaptive mindset.

Another reason that risk management is critical is that because of their complex causal relationships, time delays, non-linearities, and continuous evolution, forecasting future outcomes produced by complex adaptive systems is extremely difficult, even over relatively short periods. The challenge grows exponentially harder as the time horizon lengthens.

To be sure, there are some promising quantitative methods on the horizon for forecasting outcomes produced by complex adaptive systems (e.g., combinations of agent based modeling and artificial intelligence methods like reservoir computing). But we’re still probably quite a few years away from those methods being perfected, much less widely diffused and deployed.

That means we’re left with qualitative approaches to forecasting and decision making in complex adaptive systems, such as those used by the Good Judgment Project (described in the book “Superforecasting” by Tetlock and Gardner) or Marvin Cohen’s Recognition/Metacognition model (as described in his paper, “Metarecognition in Time-Stressed Decision Making: Recognizing, Critiquing, and Correcting”).

The former can be summed up as follows (I was on the GJP team for all four years):

• Start by looking for a base rate for the outcome you’re trying to forecast (e.g., for many business outcomes, Mike Mauboussin’s “The Base Rate Book” is an excellent source). It usually won’t be a perfect match, but it is a critical starting point.

• Then consider factors specific to the forecasting question at hand. What are the key elements in the situation? How are they related (look for non-linear relationships)? How could the interaction of these elements cause the situation to evolve, either on their own or in response to actions you or others might take? The more similar the forecasting question is to the one behind the base rate you used, the more cautious you should be about moving your situation specific forecast away from the base rate.

• Keep adjusting your forecast as you receive two types of new information. High value evidence is more likely to be observed (or not observed) if just one of your hypotheses is true. Information that triggers a feeling of surprise is a warning that your mental/forecasting model is incomplete and needs to be updated.

• After you make a forecast, always do a Pre-Mortem. Assume that it is some point in the future and it has turned out to be wrong (or your plan has failed). Write down (a) why this happened; (b) what signals you missed; and (c) what you could have done differently to avoid failure. This method works because we are much more detailed in our thinking when we seek to explain the past than when we try to anticipate the future. Use the result of your Pre-Mortem (which works even better when done in a group) to rethink your forecasting model, collect new information, and/or modify your plan.

• Last but not least, improve your predictive accuracy by combining your forecast with others, ideally ones that are based on different information and/or a different methodology.

Another aspect of what it means to be skilled in the context of complex adaptive systems is being aware of some critical aspects of human nature that make it easier to forecast the behavior of people, groups, and organizations and sometimes even larger aggregations (e.g., investor behavior).

In our experience, some of the most important include:

• To explain, predict, and remember, human beings have used stories and narratives from the earliest times. The emotions they trigger make them powerful – but also resistant to change. Hence the conventional wisdom is often lags behind the real state of affairs.

• We are naturally overoptimistic, and motivated reasoning (i.e., supporting our existing beliefs or preferred outcomes) drives our attention and often leads to overconfidence.

• When uncertainty increases, we rely more on social learning/copying and the most popular narrative, and less on the public and private information we possess. That is why as uncertainty increases (e.g., as asset class valuations approach or surpass historic highs), a limited number of narratives often grow stronger, even as they are becoming more fragile.

• Rising uncertainty also leads human beings to become more conformist, either to the views of a group or those of a strong leader. That was very adaptive deep in our evolutionary past, but is much less so today.

• In today’s world of hyperconnectivity and ubiquitous social media, and the increased complexity and uncertainty it creates, these evolutionary instincts have become supercharged.

• When an individual knows the result of a decision will be private, they tend to be risk averse. But when they know it will be public, the reverse happens (as both the decision outcome and social status will be at stake).

• When organizations are small, errors of omission (missed alarms) are often seen as more important than errors of commission (false alarms). But as organization grows, the demand for increased consistency and predictability lead to just the opposite. Hence the larger the organization, the more prone it is to being surprised.

Some Thoughts on Luck

In a society dominated by a belief in meritocracy, we tend to focus on skill as the key determinant of outcomes, and devalue the roles of system factors and luck. Except when things don’t go our way. Then it’s due to bad luck.

This is a shame, because a deeper examination of system factors and luck provides a much more complete – and healthier – understanding of the complex root causes of many of life’s outcomes.

Here’s one example. Consider a thousand people who at 21 are given $10,000 in initial capital. For each of the following twenty 29 years, each person’s capital is increased by a random draw from the same distribution of annual returns. When they turn fifty, a few people will be very rich. The majority will not. In fact, most will be below the average level of wealth. And some will have no capital left.

That is the nature of luck. To be sure, in life we draw from multiple distributions, whose draws are imperfectly correlated to varying degrees. That helps to balance out outcomes (e.g., I know some very rich people who are far below average in some other important areas of life). But it is still important to recognize the nature of what we call luck (or randomness) when it comes to accumulated advantages that are created by multiplicative processes (i.e., the “Matthew Effect”).

This is even more so the case when social influence or network effects drive those advantages (e.g., see Salganik, Dodds, and Watts’ famous paper, “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market”).

This has let researchers to conclude that there are clear limits to our ability to predict the outcomes of complex adaptive systems with high degrees of social interaction (e.g., “Exploring Limits to Prediction in Complex Social Systems”, by Martin et al). Another research project also concluded that, despite the availability today of massive amounts of data on individuals, we still cannot predict their life outcomes (“Measuring The Predictability Of Life Outcomes With A Scientific Mass Collaboration”, by Salganik et al).

To be sure, there are other kinds of luck, both good and bad, over which we have little or no control that can have both small and large consequences. A famous example occurred on December 13, 1931 when Winston Churchill looked the wrong way when crossing Fifth Avenue in New York City and was hit by a car. While injured, he survived. If he had not, the course of world history would very likely have been different. The same is undoubtedly true in the case of people who died too young.

A paradoxical thing about this kind of luck is that in some cases its impact on outcomes becomes more important as intense competition makes the level of skill among competitors more equal over time. This happens in some professional sports (e.g., hockey), and arguably active investment management. See Mike Mauboussin’s book, “The Success Equation”.

Is there anything you can do to increase your chances that both forms of luck – accumulated advantage and randomness – will work to your advantage and not your detriment?

Seven Questions to Ask Yourself

To put what follows in perspective, I graduated from college in the 1970s. In the years since, I’ve asked my friends from around the world (whose lives have taken a wide variety of paths, both intended and unintended), what questions they wish they had asked themselves when they were younger.

I make no claim that group sample is in any way representative of any population. Rather, they are a very diverse group who have experienced a wide range of ups and downs across their careers, and who have spent a lot of time reflecting upon their (often colorful) experiences. Here is my distillation of their collected wisdom:

• What activities do I most enjoy/am I relatively best at? They are usually but not always the same.

• What issues/problems/challenges most interest me? In which of these do I often find myself so immersed in that I lose track of time?

• What are the characteristics of the organizational culture in which I thrive (especially the type of boss I report to and the teammates with whom I work)?

• What kind of relationship/family do I want to have?

• Where do I want to live, and why?

• How much money do I/we need to make to have the kind of lifestyle I/we want to have over the next five years?

• What are the most important regrets I don’t want to have at the end of my life?

Inevitably there will be tradeoffs between these questions. Most people can’t have it all, or at least have it all at once. But by asking them when you’re young, you substantially increase the chances that you’ll have a sense of purpose and meaning in your life – and as a result will be a much better partner, parent, teammate, and leader.

You’ll also raise the odds building cumulative advantage, and being better able to exploit good luck and bounce back from bad luck.

Too many people don’t spend enough time thinking about these tradeoffs, and end up making them unconsciously. Trust me, you don’t want to wait to consciously confront them until a painful life event forces you to do so.

So, to sum up: You are going to live your adult life in a world of complex adaptive systems for which your education probably didn’t prepare you too well. It is easy to feel overwhelmed, and tempting to turn to popular narratives to make sense of it.

Resist that temptation. There are knowledge and skills you can learn that will enable you to thrive despite, and in some cases because of that complexity.

Above all, it helps immensely to know who you are and what you’re looking for in life. That is the path to finding purpose, meaning, and peace in our complex, hyperconnected, uncertain, and anxious world.

Good luck.