Swarms and tribes

Authors: Jonathan Grudin
Posted: Mon, March 31, 2014 - 8:03:23

A crack team led by Deputy Marshall Samuel Gerard (Tommy Lee Jones) race about in hot pursuit of Harrison Ford’s fugitive Dr. Richard Kimble. Gerard finds one of his men standing motionless. 
Gerard: “Newman,
what are you doing?!
Newman: “I'm thinking.”
Gerard stares. “Well, think me up a cup of coffee and a chocolate doughnut with some of those little sprinkles on top, while you're thinking.” Walks away.
The Fugitive (Warner Bros., 1993)

Ant colonies

Ants scurry about in a frenetic mix of random and directed activity. They gather construction materials, water, and food, and respond to threats. Ants are also busy underground, extending a complex nest, caring for the queen and her eggs, and handling retrieved materials. Decomposing leaves are artfully placed in underground chambers to heat the structure and circulate air through the passages; leaves can also be a source of food, directly or through fungus farming. Ants captured from other colonies are put to work. Foragers that are unsuccessful, even when only due to bad luck, shift to other tasks. Remarkable navigational capabilities enable ants to find short paths home and avoid fatal dehydration. Arboreal species race to attack anything that brushes their tree. Their relentless activity is genetically programmed: There are no ant academies.

Ant programming isn’t perfectly adapted to this modern world. Fire ants invading my Texas condo marched single file by the thousands into refrigerators or air conditioning units, where they were frozen or fried, sometimes shorting out an AC box. Humans rarely exhibit such unreflective behavior. Doomed military offensives such as the Charge of the Light Brigade prompt us to ask whether soldiers should sometimes question orders.

The health of the ant colony relies on the absence of reflection. It would bode ill if individual ants began questioning their genetic predispositions. “The pheromones signal something tasty that way, maybe a doughnut with sprinkles, but I don’t like the looks of that path,” or “Maybe we could come up with a better air circulation system, let’s have a committee draw up a report.” Ants don’t think, but they’re doing OK. They outnumber us. If, as seems plausible, ants are here when we’re gone, our capability for reflection could be called into question, should any creatures be around that ask questions. The ants won’t [1].


According to my favorite source, a single ant supercolony comprising billions of workers was found in 2002, stretching along the coasts of southern Europe. In 2009 this colony was found to have branches in Japan and California, no doubt enabled by our transportation systems: a global megacolony. Does it have an imperialistic plan to displace rival ant supercolonies? No, each ant follows its genetic blueprint.

We’re globalizing, too. Not long ago Homo sapiens appeared to have two supercolonies, but the bonds holding them together were less enduring than ant colony bonds. Nevertheless, we are forming larger, globally distributed workgroups. We may yet become a global megacolony. If we don’t, ants may inherit the earth sooner rather than later.

The human colony

Looking back a few thousand years, a small tribe couldn’t afford to lose many members through random behavior. If Uncle Og headed down that path and did not return, let’s think twice about going that way alone! When ants stream to their deaths, lured by false pheromone signals triggered by appliances, the colony has more where they came from. In contrast, our ability to analyze and reason enabled us to spread across the planet in small groups. A century ago we were still overwhelmingly rural—isolated and often besieged. Information sharing was limited. Each community worked out most stuff for itself. Reflection was valuable.

How are the benefits and the opportunity costs of cogitation affected as the Web connects us into supercolonies? Given a wealth of accessible information, is my time better spent searching or thinking? Tools make it easier to conduct studies; is it better to ponder the results of one, or use the time to do another study? Cut new leaves or rearrange those brought in yesterday?

Many research papers represent about three months’ work, with students or interns doing much of it. After publishing three related papers, will I contribute more by spending six months writing a deep, extensive article and carefully planning my future research—or by cranking out additional studies and two more papers?

We are shifting to the latter. Journals, handbooks, and monographs are in decline. Conferences and arXiv thrive. Arguably, we know what we are doing or our behavior is being shaped appropriately. The colony may be large and connected enough to thrive if we scurry about, cutting and hauling leaves without long pauses to reflect. Beneficial chance juxtapositions of results will simulate reflection, just as the frenzied instinct-driven construction of an ant nest appears from the outside to be a product of reflective design. The large colony requires food. For us, as for the ants, there are so many leaves, so little time.

Shifting our metaphoric social insect, the largest social networking colony is the IBM Beehive compendium: twenty-something research papers scattered across several conference series. No survey or monograph ties the studies together. I lobbied the authors to write one, but they were heads down collecting more pollen, which was rewarded by their management and the research community.

Working on a handbook chapter, I did it for them, tracking down the studies, reviewing them, and trying to convert the pollen into honey. It was hard to stitch the papers together. For example, the month and year of some work was not stated, and publication date is not definitive in a field marked by rejection and resubmission. In a rapidly evolving domain, knowing the sequence would help.

In retrospect, they were probably right about where to invest time. I found a few higher-level patterns and overarching insights, but few will take note when the handbook chapter is published next month. Social networking behaviors have moved on. The Beehive has been abandoned, the bees have flown elsewhere, leaving behind work that is now mainly of historical value, although bits and pieces will spark connections or confirm biases and be cited. From the perspective of my employer, the field, and intellectual progress, my time could have been better spent on a couple more studies.

It is ultimately a question of the utility of concentrated thought. How might we find objective evidence that scholarship is useful in this century? I’m sentimentally drawn to it, but the effort required to become a scholar might be more usefully channeled into other pursuits. The colony would collapse if ants spent time contemplating whether or not to blindly follow pheromones. Through frenetic activity they build a beautiful structure and the colony thrives. Is life in our emerging megacolony or swarm different? Race around, accept that bad luck will sideline many, and plausibly we will thrive. If an occasional false pheromone lures a stream of researchers to a sorry fate, there will be more where they came from!

The tribe and the swarm

Consciously or unconsciously, we’re choosing. Fifteen years ago, an MIT drama professor told me that with the digital availability of multiple performances, students who analyze a performance in detail do not do well. Better to view and contrast multiple performances, spending less time on each. Other examples:

  • In an earlier era, if one of the five people who were engaged in similar work performed exceptionally well, the tribe benefited by bringing them together so that the other four learned from the fifth. Today, it may be more efficient for a large organization to let the four flounder, social insect style. Successes can be shared with people working on other tasks; enough will connect to make progress. In other words, 80% conference rejection rates that were a bad idea when the community was smaller may now be viable. The community-building niche once served by conferences may be unnecessary.

    Many senior researchers disdain work-in-progress conferences—they want strong 20%-acceptance pheromone trails. If less-skilled colleagues who rely on lower-tier venues perish through lack of guidance, no matter. The as-yet-unproven hypothesis is that the research colony will thrive without the emotional glue that holds together a community.

  • When more effort was required to plan, conduct, and write up a study, it made sense to nurture and iterate on work in progress. With high rejection rates and an inherently capricious review process, researchers today shotgun submissions, buying several lottery tickets to boost the odds of holding one winner. Rejected papers may be resubmitted once, then abandoned if rejected again. Not all ants make it back to the nest, but when those that return carry a big prize, the colony thrives.

  • Any faculty member who mentors a couple successful students has trained an eventual replacement. In the past, this could mean working closely with one graduate student at a time. Today, many faculty have small armies of students, most of whom anticipate research careers. “Is this sustainable?” I asked one. “It’s a Ponzi,” he replied cheerfully. Not all students will attain their goals. In a tribe this could be a major source of discontent and trouble. Swarms are different: Foragers who fail, even when due to bad luck, take on different tasks.

The ghost in the machine

Efficiencies that govern swarm behavior may now apply to us, but there is a complication. Our programming isn’t perfectly adapted to this modern world. Our genetic code is based on the needs of the tribe. Until natural selection eliminates urges to reflect, feelings of concern for individual community members, and unhappiness over random personal misfortune, there will be conflict and inefficiency. In 1967, Ryle’s concept of the ghost in the machine was applied by Arthur Koestler to describe maladaptive aspects of our genetic heritage. The mismatch grows.

If on a quick read this is not fully convincing, you could spend some time reflecting on it, but it may be wiser to return to working on your next design, your next conference submission, and your next reviewing assignment.

1. Not all ant species exhibit all these behaviors. Some ants are programmed for rudimentary “learning,” such as following another ant or shifting from unsuccessful foraging to brood care.

Thanks to Clayton Lewis for discussions and comments.

Posted in: on Mon, March 31, 2014 - 8:03:23

Jonathan Grudin

Jonathan Grudin is a principal researcher in the Natural Interaction Group at Microsoft Research.
View All Jonathan Grudin's Posts

Post Comment


Comments submitted to this site are moderated and will appear if they are relevant to the topic.

@Paul Resnick (2014 04 02)

I think you’d like the Collective intelligence conference, Jonathan.  Not only are there presentations reflecting on models of collective intelligence for ant colonies, but there are no archival proceedings, and a very high acceptance rate. Hope to see you there!