Carol Hall, Amanda Roshier
In this article we consider the challenges involved in measuring and interpreting animal behavior. Specifically, we focus on trying to understand an animal’s responses to a new system, which might be a new computer-based system the animal interacts with or a new management system (such as housing, feeding, etc.), where the animal-computer interaction element is actually the tools used to measure the observed behavior. Reference to “measuring” behavior suggests this process is straightforward and relies solely upon recording observable actions. The interpretation of what is observed is clearly a more challenging task, one that animal behavior researchers have long grappled with using additional physiological measures and comparative approaches to attribute meaning to what is observed. However, the challenges associated with recording behavior and quantifying it in a meaningful way should not be underestimated. We present examples of issues that arise when designing a behavior-analysis tool that aims to facilitate this task. We hope that some of the experiences in observing, recording, and interpreting animal behavior that we recount here will highlight important points to consider, both when measuring behavior to evaluate interactive systems and during the design of these systems.
Why study behavior? Animals interact with their environment and other organisms to enhance their well-being and survival. From these outward expressions of behavior, inferences can be drawn to further understand biology, provide indicators of welfare or well-being and identify preferences, and provide insight into animal perspectives. By making measurements of behavior—that is, through the act of assigning values to our observations—reference values are created that permit further analysis and evaluation, thus enhancing the scientific rigor of our inquiries. The research surrounding behavior has been a multidisciplinary effort, with origins in two scientific approaches: psychology (the study of the mind) and ethology (the biological study of behavior). In the early 20th century, psychologists were concerned with elucidating the general processes of behavior under laboratory conditions; this primarily focused on the processes of learning. In contrast, ethologists were wary of experimental work and instead carried out fieldwork to gain understanding of behavior in natural contexts. As research progressed, ethology established methodology for observing and measuring behavior, whereas psychologists honed experimental design and quantification. In time, these methodologies would be shared across both disciplines, with additional contributions from other research fields including neuroscience, social science, and computer science.
The subject matters. It is time well invested to get to know our animal participants, both as individuals with their own unique life experiences, and as representative of species-specific characteristics. Species and individual characteristics should be considered broadly, recognizing both psychological and physiological components. What are their sensory abilities and how do they perceive the world? Are there any features of their behavioral repertoire that will facilitate environment manipulation? For example, dogs fetch objects with their mouths, whereas a foraging species such as the horse typically would not. That is not to say behaviors cannot be learned (or taught), but we should check if their anatomy will permit or restrict their ability to operate a system. Are they a social or solitary species? What prior learning has taken place? What motivates them? What is their personality? Has their well-being (including health and emotional state) been assessed? Does their age affect their cognitive capacity or physical ability? Will they be inquisitive or fearful of the system? An animal’s willingness to engage with a system may relate to prior experiences; have they seen this or something similar before, and was it a good or bad experience? What was their environment like during important developmental stages, such as those sensitive periods when the young animal’s nervous system is developing? Be aware of neophobia, a fear of novelty; this is an important survival strategy that can produce species-specific reactions such as freezing or running away. Although not exhaustive, this list highlights factors to consider that will shape the animal’s behavior and interaction with a system, factors that should influence the design of systems and subsequently influence how behaviors are interpreted.
What can be measured? Our human senses gather information about the external world; this information is then sent to the brain for processing, which creates perception. That is our reality of the world. With respect to gathering behavioral information, we are particularly tuned to process visual stimuli, such as features of morphology and movement. However, the level of detail we perceive depends on the capacity of our senses, any training or experience we have in recognizing these features, our ability to recall information, and even our level of fatigue. Ultimately, just how good a measurement is will depend on how it fulfills the criteria of objectivity, reliability, and validity, where objectivity aims to eliminate judgement, bias, or prejudice; reliability is how consistently a measurement can be made; and validity represents how accurately that measurement corresponds to the real world. To help us meet these fundamental criteria, the deficiencies of human perception may be enhanced by technology.
The level of detail we perceive depends on the capacity of our senses, any training or experience we have in recognizing these features, our ability to recall information, and even our level of fatigue.
Research Approaches to Measuring Behavior
Approaches used to measure behavior are many and varied and will depend upon the type of behavior the researcher is interested in, which in turn depends upon the reason for the study. Specific methods have been developed to answer specific questions. For example, a mechanistic approach, involving the measuring of different aspects of behavior independently of each other, has formed the basis of many of the quantitative methods in use [1,2]. The challenges to measuring behavior in confined, laboratory, or captive situations differ from those experienced in the field. The familiarity of the animal to human intervention, its habitat, social grouping—not to mention its size and physical features—all require careful consideration when deciding which approach to take. In addition, a study may require detailed observation of an individual, the behavior of the group as a whole, or behavioral interaction within the group, between groups, or between species.
Observing behavior. When choosing between “live” observations and using video footage, the choice is not obvious. Although video allows reruns of footage to ensure accurate recording, revisiting different individual animals, checks for inter- and intra-recorder consistency, and retention of footage for future analyses, this method does have limitations. The positioning of cameras to ensure visibility of the animals under observation, the potential for animals to move out of the frame, light conditions, and other environmental factors can result in footage that lacks sufficient detail of the required behaviors. A human observer can move to maintain a clear view, follow animals if they move away, and monitor other occurrences that may affect the observed behavior but could be off-camera. However, the human observer may influence the behavior of the animals under observation, and controlling for such effects is one of the golden rules of animal observation. In situations where the area for observation is restricted (as in cages and other animal enclosures), the best option is video footage recorded on camera(s) located to enable a view of the whole area. Once this continuous recording has occurred, a vast amount of time is needed to analyze this data.
Measuring behavior. The basis of measuring the behavior of a specific species is the development of an ethogram (i.e., a list of behaviors) that includes all features of its natural behavioral repertoire. Accurate descriptions of each behavior are listed in order to minimize the potential for variance based on subjective judgment. The ethogram includes behavioral states (ongoing, generally mutually exclusive behaviors such as sleeping, running, feeding) and behavioral events (momentary actions that may occur throughout a behavioral state or as the behavioral state changes). The duration of each behavioral state is generally recorded and the frequency of behavioral events noted. The Handbook of Ethological Methods  is an invaluable source of information about the ethological approach to the study of animal behavior.
Although there have been developments in the automation of video analyses, the systems currently available do not provide sufficient behavioral details in the majority of species, and human input is still required. Video-tracking software is used to monitor the behavior in laboratory rodents, in particular. A recent development is the inclusion of mice and rat behavior recognition (e.g., grooming, sniffing behavior) as well as the monitoring of activity and movement patterns. Such automated behavior recognition reduces the time required to analyze behavior and removes the risk of human error. In laboratory species predominantly used for research within neuroscience, this is a valuable, time-saving tool. Currently, in the majority of species, behavioral analysis requires designing an ethogram and either inputting this into a behavioral analysis package or scoring the behavior by hand. Some packages require the development of behavioral codes to use when scoring behavioral footage and produce extensive analyses of the data.
Unfortunately, whichever process is chosen, the time needed to view and record the behavior is likely to be greater than the duration of the footage. Very often, specific behaviors are of interest to the observer, for example, agonistic or affiliative animal interactions. A system that could identify such occurrences within the overall footage would speed things up considerably. Once identified, the human observer could record the nature of the interaction and the individuals involved, before fast-forwarding to the next occurrence of interest. Unless there is behavioral synchrony within a group of animals, this will need to be repeated for each individual animal. This brings us to the challenge of identification.
Identifying individuals within a group of animals is often problematic. In a flock of several hundred sheep, which features differentiate individuals? In domestic/laboratory animals, colored markers or symbols could be identifiers. This is not usually a possible or desirable method of identification in wild animals. Instead, researchers need to familiarize themselves with individual animals until they can reliably identify them. However, certain species have markings unique to individuals, making automatic recognition possible. For example, an automated recognition tool uses a barcode-scanning approach to scan the stripe patterns of zebra. This method offers the potential for identifying individuals in other species with patterned markings, but it may falter in the case of animals with more uniform coloration.
In some studies, features of group behavior rather than individuals interest the researcher. The identification of individuals is still necessary, but it could be the role of each animal, the spatial distribution of individuals, the distance between different groups, or any number of factors relating to group behavior that need to be measured. Group composition can be assessed by recording individual characteristics such as gender, age, and familial ties. Intra-group interactions may provide information about hierarchical structure and competition for resources. Measuring the distance between individuals can indicate affiliative ties, either by approximation (often using the animal’s body length as a guesstimate) or by using a rangefinder (which measures the distance of a target object from the observer and facilitates the calculation of distances between individuals). Overall, social network analysis is a means of exploring spatial and other relationships among group members.
When observation is not possible. Some aspects of animal behavior may be hard to observe either live or from video footage taken from a static camera. For example, ecologists are interested in the behavior and movement of animals within the environment. In some cases, these movements are across great distances, in water, in the air, or even underground. By using animal-borne video and environmental data collection systems, video footage from the animal’s point of view is combined with data from other sensors (to monitor factors such as location and temperature). Attaching such devices to an animal requires the design of appropriate equipment, catching the animal, a means of attachment that is secure but does not restrict the animal’s behavior, and a way of removing and retrieving the equipment, not to mention an effective means of accessing the data. Monitoring movement patterns of animals with GPS tracking devices has become common practice, with equipment designed for hedgehogs, vultures, sharks, and others. Advancing technologies enable continuous recording for longer periods, potentially for many months. Pedometers/accelerometers provide information about activity and time spent lying down, moving, feeding, and so on. When large numbers of animals are involved and the interest in their behavior is predominantly commercial, as in dairy cows, this equipment provides information about changes in activity that may relate to the onset of estrus, calving, or health problems.
Behavior and learning. Behavior changes in response to training provide measures of intelligence, learning ability, and the effectiveness of the training method used, or of the trainer. Learning trials designed to assess intelligence and perceptual ability, as well as training animals to fulfill specific human-oriented roles, all result in measurable changes in behavior. The speed of any behavioral change, the accuracy of performance, and the number of errors made are all ways of assessing learning. In human-derived tasks, a set criterion for successful learning, or a behavioral response to a specific signal, indicates that the animal has learned the task. In their natural environment, animals adapt their behavior in response to environmental cues or other animals, and according to the consequences of their actions. This association between behavior and outcome forms the basis of adaptive behavior and learning theory. The use of computer-assisted tasks, interactive screens, and software developed to assess performance has resulted in recent animal-learning studies based on animal-computer interactions. While this development is inevitable, designing tasks and equipment with the animal species in mind is imperative. In the past, an anthropocentric approach to assessing animals has often resulted in an underestimation of their intelligence; in fact, species-specific abilities and perception form the basis of any fair intelligence test.
Interpreting behavior. Using the objective, mechanistic approaches to measuring behavior described here leaves us with the problem of how to interpret observations and their measurements. Sequences of behavior—what behavior generally follows or precedes another behavior—can indicate a particular behavior’s meaning. For example, if a horse approaches another with its ears back, then lunges forwards with its head and neck, then goes to bite the other, this is probably aggression. If this sequence of behavior always occurs after ears-back behavior, then ears back takes on a specific meaning. But the meaning of behavior is not always clear. Facial expression in some species is a good indication of how an animal is feeling and likely to behave. However, facial expression may be deceptive and does not always reflect inner feelings. Humans in particular are able to generate facial expressions at will. Such deception may be exposed by monitoring changes in circulatory patterns, particularly in the area of the face, that occur as part of an emotional response. Changes in peripheral blood flow result in changes in body-surface temperature; the heating or cooling of specific facial areas such as the nose and area surrounding the eye are measurable using infrared thermography. This technique offers some insight into emotional responses and the feelings of humans and other animals, but in species with facial hair, measuring appropriate temperature changes is fraught with difficulty.
A “whole animal” approach may take us closer to understanding what behavior means. With empathy, humans may be able to judge how an animal is feeling and what its behavior indicates. The system of free-choice profiling, whereby observers describe the animal’s interaction with its environment, qualitatively aims to assess the whole animal ). This approach may still result in misinterpretation and is likely to be less effective in unfamiliar species. The whole is certainly greater than the sum of its parts, and the more mechanistic approach has not yielded all the answers. Offering animals choices and allowing them some control over their environment may add to our understanding. Perhaps someday it will be possible to put on the virtual reality helmet and become an animal for a day. This may for many be a salutary experience.
System designers: Please do not be put off by the challenges ahead. A solution is out there! We hope that sharing our experiences in observing, recording, and interpreting animal behavior will help get us a little closer to finding it. Such a solution can play a fundamental role in understanding animals, their needs, and how to design interactive systems appropriate for them. We await the emergence of an automated (accurate!) behavior-assessment tool with great anticipation.
Carol Hall is a reader in equitation science, with research interests in horse behavior and welfare. She is a member of the International Society for Equitation Science, is on the board of the National Equine Welfare Council, and is a qualified riding instructor (British Horse Society Intermediate Instructor). firstname.lastname@example.org
Mandy Roshier is an assistant professor of anatomy and behavior. Her research interests include biomedical engineering, biomechanics, animal communication, human-animal interactions, and animal behavior in veterinary medicine. She is a committee member of the British Veterinary Behaviour Association. email@example.com
©2016 ACM 1072-5220/16/07 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2016 ACM, Inc.