November 5, 2021

Elite Underproduction

We are bad at nurturing the people who keep the frontier open.

Elite Underproduction

Elite Overproduction is a meme started by Peter Turchin. It goes something like;

Turchin noted, ... that the U.S. has far more lawyers, MBA holders, and millionaires per thousand population than it did 30 years ago. This is worrying because “elite overproduction” is, according to Turchin, an underlying cause of political instability. He blames elite overproduction for playing a role in the fall of Rome and the American Civil War, for instance.

It's also essentially unworkable on its own as an explanatory social theory, which Turchin himself goes on to clarify;

[Turchin] says “stagnating and declining living standards of the general population and increasing indebtedness of the state” are two other critical factors.

In other words, rather than some inability of 'power structures' to 'absorb aspiring elites', the underlying material conditions of society degrade to the point where socio-political upheaval is inevitable. Turchin has cause and effect the wrong way around, Elite Overproduction is a consequence of a closed frontier.

Elite Overproduction is a result of Malthusian Ideology; a set of beliefs that at root reflect the idea that all frontiers are closed, and all that is left is to decide who is to own what, and who is to rule. It is the consequence of believing that things can only get better for some at the expense of others, rather than for all together. In the bloodless language of economics, in equilibrium all surplus is competed away, in the bloody language of history you instead find words like Lebensraum.

In our society, Elite Overproduction can be explained as a form of Malthusian social competition where only so many resources and only so much status is available, and the only way to access a satisfactory slice is to compete within the power structures that control them. In turn, our institutions evolve mainly to facilitate this competition rather than performing their ostensible material functions. In turn this causes resentment to grow among those who never had access to those institutions in the first place.

This situation is a consequence of Elite Underproduction. Our society is chronically bad at producing and nurturing true elites, the individuals who are capable of creating and sustaining frontiers. That makes malthusian competition inveitable, and the only thing going - the kinds of 'elites' we end up producing are good at that kind of competition and not much else.


In my line of work I meet plenty of lapsed physicists. These are people with physics degrees, even physics Ph.D's, who don't do physics. Instead they're usually some sort of programmer, or occasionally some version of a quant. Many of them work with machine learning. What they're not doing is advancing the human understanding of the physical world.

In the long run, creating more understanding of the physical world is probably more universally valuable to human flourishing than the far more parochial 'make money number go up' type of work lapsed physicists slinging JavaScript in the MAGMA[1] mines are doing for us now. If nothing else, it creates new and more interesting ways of making number go up.

Most lapsed physicists I've spoken to didn't start out pursuing physics in order to do this kind of work. Maybe they were too embarassed to admit to me that what they really wanted to do as a kid was find a hundredth-of-a-percent edge for some hedgy's algotrading, but mostly they've told me they were fascinated by what we didn't yet know about the universe, inspired by those who had gone before, and wanted to test themselves against the really difficult problems.

Young people going into physics are fundamentally ideologically driven. So why do they lapse?

How is this deep drive to push forward the frontier overcome?

Some decide quite early that a physics education isn't for them. Some decide that their colleagues are far more brilliant than they, and so they have no hope of making any meaningful contribution. A large number see significantly better economic opportunity in industry than in academia.

All of these reasons point to failure modes in our systems of scientific production.

Our apparatus of scientific education, especially mathematics education, is deeply flawed. It takes too long to bring young people to the research front, and it produces a false view of what doing science is really like. It's rational that many people decide early on that it's not for them, even though our system of scientific education in no way resembles the actual practice of scientific research.

The centralizing forces of the academy as it exists today constrain productive niches; only a select few niches are preordained as productive, and comparative advantage means their more 'brilliant' (i.e., niche-suited) colleagues will occupy them. It's rational to do something else when you see no possibility to do meaningful work.

Exiting the academic meat-grinder is rational; faced with a choice between living off ramen while being miserable chasing grant money, or getting paid while being miserable fighting with Python package managers, becoming a 'data scientist' is a reasonable choice.

Obviously, some people do make it through. Some people get lucky and some people are well suited to the process. Some manage to retain their ideological drive in spite of everything. Physics continues advancing, however slowly, in however tiny a number of correlated directions. But the opportunity cost is gargantuan.

One thousand, perhaps ten thousand times as many people could be making meaningful contributions to our understanding of the physical universe.

Similar processes and factors exist among all the natural sciences, in mathematics, as well as engineering and other adjacent fields. Our systems waste enormous potential, mainly in the causes of legibility and standardization. We are squandering the strongest ideological drives of our young people. There are many more 'elites' in terms of meaningful human progress available than we are making use of.

If we are to pursue the infinite frontier, this must change. We must reorganize scientific production to harness human drives. We cannot afford elite underproduction any longer. The stakes are too high and we are not even trying.


[1] Meta, Apple, Google, Microsoft, Amazon.