Every new technology has reshaped the economy and led to loss of some jobs and creation of others. But the technology of large language models (LLMs) is different – this time, the highly trained workers may be the ones losing their jobs. This post provides an overview of the August 2023 paper GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models which explores the impact that LLMs will have on jobs in the United States. Note that because the paper focuses on LLMs specifically, it doesn’t take into account recent advances in robotics.

Defining LLM Exposure

The paper frequently uses the term “LLM exposure” rather than “job loss from LLMs” but for understanding purposes, they are basically the same. The paper defines the term “LLM exposure” as “a proxy for potential economic impact without distinguishing between labor-augmenting or labor-displacing effects.” In other words: LLM exposure means an LLM can do that job entirely, or “augment” it. And if an LLM can do a job or augment it, we’ll need fewer people doing that job…So, “LLM exposure” is a euphemism for “job loss from LLMs.”

Evaluation Process

The paper develops a rubric to evaluate the overlap between LLM capabilities and tasks associated with specific jobs, leveraging the O*NET database which describes different jobs. The research used judgements of humans and LLMs themselves to estimate the impact of LLMs on different jobs. One caveat is that the humans involved had to be familiar with LLM capabilities, but they didn’t necessarily have to be knowledgeable about tasks performed in all possible occupations.

In general, the human raters and the LLMs generally agreed about amount of “LLM exposure” by occupation.

Nearly Everyone Will Be Affected by LLMs

Nearly everyone will be affected in some way by LLMs: 80% of workers have a job with at least 10% of its tasks exposed to LLMs.

19% of workers will be severely affected: 19% of workers belong to a job that could be half-automated (i.e., 50% of that job’s tasks are “LLM exposed”).

The results are even more alarming if other generative models and associated technologies are considered (e.g., models that can also use images). In this case, the authors estimate that half of workers could have half or more of their tasks “exposed.”

Skilled Workers Will Be Most Affected

In the past, technological progress generally raised the demand for skilled workers and lowered the demand for unskilled workers. LLM technology will likely do the opposite: workers facing higher barriers to entry in their jobs will likely experience more LLM exposure. The workers with the most training generally will have the most LLM exposure – in other words, we may end up with a situation where there are a lot of highly educated, highly trained people who’ve lost their jobs to LLMs.

The authors came to this conclusion by analyzing LLM exposure in the context of “Job Zones.” A “Job Zone” approximately measures the amount of education/experience/training needed to do a job. They found higher Job Zone, i.e. more training needed to do the job, is associated with more LLM exposure. On average, the longer it takes a human to get trained to do a job, the easier it is for an LLM to automate that job. People with Bachelor’s, Master’s, and professional degrees are more exposed to LLMs than those without higher education credentials. Higher wages are also associated with higher exposure.

One caveat to this is that when looking at skills, rather than whole jobs, skills in science and critical thinking were strongly negatively associated with LLM exposure, meaning, the results suggest that it’s harder for LLMs to automate specifically science and critical thinking skills.

Writers/Programmers Are Most At Risk

The authors found that programming and writing were strongly positively associated with LLM exposure. This makes perfect sense, as both programming and writing involve producing text as the primary goal and that’s exactly what LLMs are best at.

The specific jobs with highest estimated exposure (most likely to be automated by LLMs) are: authors, poets, lyricists, creative writers, proofreaders, copy markers, news analysts, reporters, journalists, legal secretaries, interpreters and translators, survey researchers, animal scientists, public relations specialists, mathematicians, tax preparers, financial quantitative analysts, web and digital interface designers, correspondence clerks, blockchain engineers, court reporters and simultaneous captioners, accountants, auditors, clinical data managers, and climate change policy analysts (see Table 4).

Interestingly, potential exposure to LLMs wasn’t correlated with current employment levels – so just because a job is stable now doesn’t mean it’ll be stable in the future.

Jobs Involving Physical Labor Are Least At Risk

Occupations with no “exposed tasks”, i.e. occupations least likely to be automated by LLMs are (Table 11): agricultural equipment operators, athletes and sports competitors, automotive glass installers and repairers, bus and truck mechanics and diesel engine specialists, cement masons and concrete finishers, short order cooks, hand cutters and trimmers, oil and gas derrick operators, dining room and cafeteria attendants and bartender helpers, dishwashers, dredge operators, electrical power-line installers and repairers, surface mining excavating and loading machine and dragline operators, floor layers, foundry mold and coremakers, brick masons, block masons, stonemasons, tile and marble setters, carpenters, painters, paperhangers, plasterers, stucco masons, pipe layers, plumbers, pipefitters, steamfitters, roofers, meat and fish cutters and trimmers, motorcycle mechanics, paving surfacing and tamping equipment operators, pile driver operators, metal pourers and casters, rail-track laying and maintenance equipment operators, refractory materials repairers, mining roof bolters, oil and gas roustabouts, slaughterers and meat packers, stonemasons, tapers, tire repairers and changers, and wellhead pumpers.

This paper does not consider any recent advances in robotics. For example, this new Mobile Aloha robot by Google can cook a three-course meal, do the dishes, and help with laundry.

LLMs as General-Purpose Technologies

The authors propose that LLMs are general-purpose technologies similar to the printing press or steam engine. General purpose technologies have three characteristics:

  • Continuous improvement over time;
  • Pervasiveness throughout the economy/widespread proliferation;
  • Ability to lead to complementary innovations.

The AI literature definitively demonstrates that LLMs satisfy the first criterion – they are certainly improving dramatically over time.

The paper’s results suggest that the latter two criteria also likely apply – LLMs will probably become pervasive throughout all aspects of the economy, and will lead to many complementary innovations that integrate LLMs into different industries. These complementary innovations are costly and time-consuming to produce, and may involve discovering new business models, but are likely inevitable as companies scramble to capitalize on the full potential of AI.

Out-of-the-box LLMs can already impact a large proportion of workers and jobs, but the additional complementary innovations integrating LLMs into different workflows could have an even bigger impact on workers. Maximal LLM impact will occur when LLMs are maximally integrated with larger systems. LLMs further become more powerful with access to tools: “while it’s true that on its own GPT-4 does not know what time it is, it’s easy enough to give it a watch.” It’s possible that we may end up in a situation where LLMs can execute any task typically performed at a computer.

This won’t happen all at once. There may be phases, for example, in which certain jobs first become more precarious and unstable (e.g. transitioning from full time to freelance) before those jobs get fully automated.

Risks of Social Unrest and Human Enfeeblement

The paper does not speculate on the downstream consequences of such disruptive effects of LLMs on the job market, but here are a couple consequences worth considering:

  • Mass Unemployment: In their excellent paper about catastrophic risks of AI, Hendrycks et al. point out that AI could lead to massive job losses. AI systems could work 24 hours a day, be copied many times, run in parallel, and process information way more quickly than any human being ever could. Societal inequality could increase, and people could become dependent upon the owners of AI systems.
  • Social Unrest: If a huge number of people suddenly lose their jobs, this could lead to social unrest.
  • Human Enfeeblement: Hendrycks et al. also point out that if we eventually use AI systems to replace all workers, then humans may become completely reliant on AIs. Humans could gradually give over more and more jobs and skills to AI systems, and society could eventually reach a point where humans don’t have any knowledge or skills and don’t understand how their own civilization works.

Conclusion

I like to end posts on a positive note, but I’m genuinely concerned about the impact that AI will have on the labor market. Modern AI systems are astonishing feats of ingenuity with incredible capabilities – and these incredible capabilities are precisely why these AI systems will be so disruptive. Everyone should stay as informed as possible. And, workers in the highest-risk jobs may want to consider gaining complementary skills in areas that are less likely to be affected by AI advances.

Featured Image

The featured image was adapted from Rosie the Riveter, Wikipedia, public domain