ACM Interactions

“To become more than a sergeant? I don't consider it. I am a good sergeant; I might easily make a bad captain, and certainly an even worse general. One knows from experience.”
   — from Minna von Barnhelm, by Gotthold Ephraim Lessing (1729–1781)

“There is nothing more common than to hear of men losing their energy on being raised to a higher position, to which they do not feel themselves equal.”
    — Carl von Clausewitz (1780–1831)

"All public employees should be demoted to their immediately lower level, as they have been promoted until turning incompetent."
   — José Ortega y Gasset (1883–1955)

“In a hierarchy individuals tend to rise to their levels of incompetence.”
   — Laurence Peter (1919–1990)

We should be enjoying a golden age of competence. We have easy access to so much information. YouTube videos show us how to do almost anything. Typing a question into a search engine very often retrieves helpful answers. We see impressive achievements: Automobiles run more efficiently and last longer, air travel grows steadily safer, and worldwide distribution of a wide variety of products is efficient. Nevertheless, there is a sense that overall, the world isn’t running that smoothly. Governments seem inept. One industrial sector after another exhibits bad service, accidents, inefficiencies, and disastrous decisions. The financiers whose ruinous actions led to worldwide recession and unemployment didn’t even lose their jobs. In HCI, many nod when Don Norman says “UI is getting worse—all over.” How could incompetence be on the rise when knowledge and tools proliferate?

The last of the four opening quotations, known as the Peter Principle, was introduced in the 1969 best-selling book of the same name. The other writers noted that people are promoted to their levels of incompetence; Peter went further, explaining why organizations keep incompetent managers and how they avoid serious harm. I will summarize his points later, but for now join me in a thought exercise:

Assume the Peter Principle was true in 1969. How are technology and societal changes affecting it?

There are several reasons to believe that managerial incompetence is escalating, despite the greater capability of those who are competent—who, in Peter’s words, “have not yet reached their levels of incompetence.”

Strengthening incompetence

1.  Competent people are promoted more rapidly today. Thus, even if well-trained, they can reach their levels of incompetence more quickly. In the rigid hierarchical organizations of the past, promotions were usually internal and often within a group. Few employees had to wait the 62 years and counting that Prince Charles has for his promotion, but wait they did. Today, with the visibility that technologies enable, competent employees can easily find suitable openings at the next higher level in the same or a different organization. Organizational loyalty is passé. A software developer joins a competitor, an assistant professor jumps to a university that offers immediate tenure, a full professor is lured away by a center directorship or deanship. The quickest way to advance in an organization can be to take a higher position elsewhere and return later at the higher level. LinkedIn reduces the friction in upward trajectories.

2. Successful organizations grow more rapidly than they once did, creating a managerial vacuum that sucks people upward. Enterprises once started locally and grew slowly. Mass media and the Internet enable explosive growth, with technology companies as prime examples. As a project ramps up and adds team members, experienced workers are incented and pressured to move up a management ladder that can quickly grow to 8 or 10 rungs. A person can plateau at his or her level of incompetence while very young.

3. The end of mandatory retirement extends the time that employees can work at their levels of incompetence. In 1969, Peter’s great teacher who became an incompetent principal probably had to retire at 65. Today he could have a decade of poor performance ahead.

4. The decline of class systems and other forms of discrimination is terrific, but egalitarian systems are less efficient if everyone progresses to their level of incompetence, whereas competent employees trapped beneath a class boundary or a glass ceiling are ineligible for promotion and thus fail to achieve incompetence. In the 1960s, many women found job opportunities only in teaching, nursing, and secretarial work. Accordingly, there were many extraordinarily capable teachers, nurses, and secretaries. I benefited from this indefensible discrimination in school and my father benefited from it in his job. (If this argument seems alarming, read to the end!)

5. Increased job complexity is a barrier to achieving and maintaining competence. As the tools, information, and communication skills required for a job increase, someone promoted into the position is less likely to handle it well. The pace of change introduces another problem: A competent worker could once count on remaining competent, but now many skills become obsolete. “Life-long learning” isn’t a cheerful concept to someone who was happy to finish school 30 years ago.

Accept the premise of the Peter Principle and these are grounds for concern. But you may be thinking, “The Peter Principle is oversimplified, competence isn’t binary, lots of us including me haven’t reached our levels of incompetence and don’t plan to.” Peter would disagree and insist that you are on a path to your level of incompetence, if you haven’t reached that destination already. I will summarize Peter’s case, but first let’s consider another possibility: Do other changes wrought by technology and society undermine the Peter Principle? The answer is yes.

Weakening the Peter Principle

1. Technology has so weakened hierarchy in many places that it’s difficult to realize how strong hierarchy once was. Peter christened his work “hierarchiology” because flat organizations are not built on promotions. The ascent at the heart of his principle is almost inevitable in rigid hierarchies where most knowledge of a group’s functioning is restricted to the group. I worked in places where initiating a work-related discussion outside the immediate team without prior managerial approval was unthinkable. Memos were sent up the management chain and down to a distant recipient; the response traveled the same way. The efficiency and especially the ambiguous formality of email broke this. A telephone call or knock on the door requires an immediate response; an email message can be ignored if the recipient considers it inappropriate to circumvent hierarchy. Studies in the 1980s showed that although most email was within-group, a significant amount bypassed hierarchy. Hierarchy is not gone, but it continues to erode within organizations and more broadly: Dress codes disappear, children address adults by first name, merged families have complex structures, executives respond directly to employee email, and everyone tweets.

2. Hierarchy benefits from an aura of mystery around managers and leaders. Increased transparency weakens this. In hierarchical societies, rulers tied themselves to gods. Celebrities and the families of U.S. Presidents once took on a quasi-royalty status. In The Soul of A New Machine, the enigmatic manager West was held in awe by his team. Not so common anymore. Leaders and managers are under a media microscope, their flaws and foibles exposed [1]. When managerial incompetence is visible, tolerating it to preserve stability and confidence in the hierarchy is more challenging. In addition, internal digital communication hampers an important managerial function: reframing information that comes down from upper management so that your unit understands and accepts it. The ease of digital forwarding makes it easier to pass messages on verbatim, and risky to do otherwise because a manager’s “spin” can be exposed by comparison with other versions.

3. When organizations are rapidly acquired, merged, broken up, or shut down, as happens often these days, employees have less time to reach their levels of incompetence. Unless brought in at too high a level, they may perform competently through much of their employment.

And the winner is…

…hard to judge definitively. We lack competence metrics. People say that good help is harder to find and feel that incompetence is winning, perhaps because we expect more, promote too rapidly, or keep people around too long. But could it be that only perceived incompetence is on the rise? Greater visibility and media scrutiny that reveal flawed decisions could pierce a chimera of excellence that we colluded in maintaining because we wanted to believe that capable hands were at the helm.

Despite these caveats, I believe that managerial incompetence is accelerating, aided by technology and benign social changes that level some parts of the playing field. Two of the three counterforces rely on weakened hierarchy, but hierarchical organization remains omnipresent and strong enough to trigger hierarchy-preserving maneuvers at the expense of competence, as summarized below.

Part II: Hierarchy considered unnatural

Peter’s “new science of hierarchiology” posits dynamics of levels and promotions. Archaeology and history [2] reveal that when hunter-gatherers became food-sufficient, extraordinarily hierarchical societies evolved with remarkable speed: Egypt and Mexico, China and Peru, Rome and Japan, England and France. Patterns of dysfunction often arose, but hierarchy persevered. Our genes were selected for small-group interaction; large groups gravitate to hierarchy for social control and efficient functioning. Hence the universality of hierarchy in armies, religions, governments, and large organizations.

As emphasized by Masanao Toda and others, we evolved to thrive in relatively flat, close-knit social organizations where activity unfolded in front of us. Hierarchical structures are accommodations to organizing over greater spatial and temporal spans. They can be efficient, but because they aren’t natural we should not be surprised by dysfunction. Hierarchy that emerges from our disposition to jockey for status in a small group can play out in less than optimal ways in large dispersed communities.

Those at the top work to preserve the hierarchy, with the cooperation of others interested in stability and future promotion. When employees are promoted but prove not up to the task, removing them has drawbacks. It calls into question the judgment of higher management in approving the promotion. Who knows if another choice will be better? The person’s previous job is now filled. If it is not disastrous, best to leave them in place and hope they grow into the job. In this way, a poor school superintendent who was once a good teacher or athletic coach hangs on; an incompetent officer is not demoted. When high-level incompetence could threaten an organization, other strategies are employed: An inept executive focuses on procedural aspects of the job and is given subordinates “who have not yet risen above their levels of competence” to do the actual work.  An incompetent manager is “kicked upstairs” to a position with an impressive title and few operational duties. Peter labels this practice percussive sublimation and describes organizations that pile up vice presidents “on special assignments.” In a lateral arabesque, a manager is moved sideways to a role in which little damage can be done. Another maneuver is to transfer everyone out from under a high-level non-performer, yielding a free-floating apex. Reading Peter’s amusing examples of these and other such practices can bring to mind a manager one has known. Perhaps more than one [3].

The Peter Principle

Researching a book on the practices of good teachers, Laurence Peter encountered examples of poor teaching and administration. His humorous compilation of “case studies” drawn from education and other fields, fictionalized and padded with newspaper stories, eventually found a publisher and became a best-seller. A 1985 book subtitled “The Peter Principle Revisited” promised “actual cases and scientific evidence” behind “the new science of hierarchiology.” It delivered no such thing. He may have observed and interviewed hundreds as claimed, but he provides a limited set of examples: capable followers promoted to be incompetent leaders, capable teachers who made poor administrators, experts on the shop floor who became bad supervisors, great fundraiser-campaigners who prove to be poor legislators, and so on. Sources of eventual incompetence are intellectual, constitutional, social, and other mismatches of skill set to position requirements.

The phenomenon is also evident in less formal hierarchies. A good paper presenter is promoted to panel invitee. A successful panelist receives a keynote speech invitation. A young researcher is invited to review papers. Promotions to associate editor or associate program chair, editorship or program chair, and more prestigious venues or higher professional service can follow until incompetence is achieved. Percussive sublimation and lateral arabesques are found in professional service as well as in organizations. The visibility of competent performance can undermine it by spurring invitations: A strong, proactive conference committee member may deliver weak, reactive service when subsequently on four committees simultaneously.

At times Peter claims that there are no exceptions to his principle. Pursuit of universality led him to dissect apparent exceptions, yielding the insights into how organizations handle high-level poor performers. Elsewhere Peter acknowledges that many people work ably prior to their “final promotion,” suggests a few ways to avoid promotion to your level of incompetence, and presents the nice class boundary analysis that identifies pools of competence.

The prevalence of class systems explains why the 18th and 19th century quotations above described the possibility of promotion to incompetence whereas the 20th century quotations stated its inevitability in an age with less discrimination. Less discrimination against white males, anyway. Allow me an anecdote: As CSCW 2002 ended, I went to a New Orleans post office to mail home the bulky proceedings and other items. As I started to box them, one of two black women behind the counter laughed and told me to give them to her, whereupon she discussed the science of packaging while rapidly doing the job and entertaining her colleague with side comments. I left with no doubt that the two of them could have managed the entire New Orleans postal service. Courtesy of workplace discrimination perhaps, I had the most competent package wrapper in the country.

Spending a career with a single employer—actors in the studio system, athletes and coaches with one team, faculty staying at one university, reciprocal loyalty of employees and company—was once common. Promotions were internal, waiting for promotions was the rule, and years of competent performance was common, abetted by glass ceilings and early retirements. Those days are gone.

The versatility of programming made it a nomadic profession from the outset. When I worked as a software developer in the mid-1980s, we questioned the talent of anyone who remained in the same job for more than three years. A good developer should be ready for new challenges before an opening appeared in their group, so we were always ready to find a job elsewhere. When after two years I left my first programming job—work I loved and was good at—to travel and take classes, my manager tried to retain me by offering to promote me to my level of incompetence—that is, he offered to hire someone for me to manage.

With job opportunities in all professions visible on the Internet and intranets, a saving grace of the past disappears: When the number of competent employees exceeded the number of higher positions, not all could be promoted. Today, a capable worker aspiring to a higher position can likely find an employer somewhere looking to fill such a position. 

Concluding reflections

This essay on managerial efficacy began with the observation that rapidly accessed online information is a powerful tool for skill-building: In many fields, individual competence and productivity has never been higher. Because someone who does something well is a logical candidate for promotion to manage others doing it, this undermines managerial competence: Managing is a complex social skill that is learned less when studying online than through apprenticeship models.

As class barriers and glass ceilings are removed, subtle biases continue to impede promotion, so by the logic of the Peter Principle, past victims of overt discrimination are especially likely to be capable as they more slowly approach their final promotion.

What should we do? Think frequently about what we really want in life, and keep an eye on those hierarchies in which we spend our days, never forgetting that they are modern creations of human beings who grew up on savannahs and in the forests.

Endnotes

1. “It is the responsibility of the media to look at the President with a microscope, but they go too far when they use a proctoscope.”— Richard Nixon

2. Charles Mann’s 1491 provides incisive examples and a thoughtful analysis.

3. The reluctance of monarchs to execute other royalty reflected the importance of preserving public respect for hierarchy. The crime of lèse-majesté, insulting the dignity of royalty, was severely punished and remains on the books in many countries. Today we are reluctant to prosecute or even force out financial executives who made billions driving our economy into ruin. Our genes smile on hierarchy, our brains acquiesce.

Jonathan Grudin