XIX.4 July + August 2012
Page: 19
Digital Citation

Interaction with the dirty, dangerous, and dull

Clint Heyer, Kristoffer Husøy

The environments for which interaction designers design tend to be rather benign: the office, the home. The settings are familiar to us, identifiable and relatable, often places we’ve experienced firsthand. A host of assumptions are implicit in these settings that affect the style of design inquiry and the forms of prototyping and evaluation. For example, we might assume a stable and safe physical environment, or that if something doesn’t work, it is an annoyance rather than a life-or-death situation.

In our work, we have explored quite a different context: that of an oil and gas facility, where work is at times dirty, dangerous, and dull. We describe how we approached design for this unfamiliar environment and outline some of our concepts. In our general program of research, we are exploring new, yet “close to market,” artifacts and systems for use in oil and gas, potentially for productization.


High-Octane Work

Here we focus on the industrial environment of oil and gas production and refining. Raw materials are pumped up at offshore facilities and shipped to onshore refineries by ship or through pipelines. There, refining takes place to make products such as diesel, gasoline, and methanol. Our work is informed by observational field studies, mostly conducted in Norway, but also in India.

With much of the materials highly combustible or poisonous, and areas of the plant containing extremely high temperatures or high pressure, this is understandably a high-risk workplace. Field operators carry out observation rounds and general plant maintenance behind an array of protective measures: helmet, flame-retardant clothes, gloves, safety goggles, and earmuffs. While aiding safety, this equipment impedes movement, awareness of the surroundings, and face-to-face communication with colleagues.

Unlike domestic or office environments, the oil and gas setting poses risks to human health, infrastructure, and the natural environment. If things go bad, they can go really bad. As in other critical facilities, such as chemical plants and healthcare, there are stringent requirements regarding safety, dependability, and accountability for work activity and the systems utilized. These requirements, coupled with the harsh outdoor environment typical of many plants, necessitate a restricted palette of technology that can be used in potential solutions.

Field operators carry out a broad range of manual tasks and operations, engaging with the plant in a direct, tangible manner. They scramble, crawl, climb, and weave through and on the plant; squeeze, turn, dip, disassemble, press, and pull objects. Grease-covered visages and the occasional pulled muscle are testament to the degree of physicality in everyday work at the plant. The plant also “talks back” to the operator, with sound, smell, vibration, and light emitted by equipment and as a by-product of plant processes. For example, operators can gauge efficiency of one process by looking at the furnace flame color, estimate ball-bearing condition by placing their hand on a motor housing, or smell for machine fluid leaks. When unscrewing nuts, operators form an awareness of when the nut was last moved, based on the physical “give” of the nut as well as visual cues such as patina or fresh scratches. This awareness is folded into activity, for example, applying more pressure if the nut is tight, or perhaps giving reason for pause if the operator unexpectedly finds himself opening a valve that has long lain dormant. Operators thus have a rich experience or “feel” of the plant that is strongly situated in physical activity and the environment and report that this feeling is important for maintaining situational awareness, diagnosing faults, and determining what corrective action to take.


In order to sensitize ourselves to the particular setting and develop a deeper understanding of how work is practically accomplished, we visited a number of oil and gas sites, observing people as they went about their work and conducting informal interviews [1,2]. Visiting such sites has some pragmatic difficulties: access control is stringent, and safety courses must be completed. Common tools in qualitative fieldwork, such as voice recorders and cameras, cannot be used due to the risk of explosion, unless additional portable gas detectors are used and special permits acquired.

Designing for industrial environments imposes a number of constraints necessary to meet government regulations and policies of the operating company. Industrial equipment is built to adhere to various international standards, based on the device’s susceptibility to dust, moisture, dropping, and so forth. Usually this results in environmentally sealed surfaces and plenty of bulky protective casing. Input devices for industrial equipment tend to be things such as membrane keyboards, large buttons, and resistive touchscreens. In the oil and gas industry, there are additional requirements to ensure safe operation in explosive environments, for example, ensuring a device is able to contain internal sparks or explosions (such as from a battery) and be neutral to external sparks. As a result, artifacts designed for these environments tend to be large, heavy, and expensive. Even Ethernet cables, which may seem rather innocuous, need to be specially constructed, shielded, and made of particular materials in order to render them safe for use in an oil and gas plant.

To inspire design and broaden perspective beyond the status quo of industrial equipment, we used the technique of analogous practices [3]. Noting the parallels in work activities and organizational structure between healthcare and oil and gas, we designed a series of provocative concepts inspired by the healthcare setting: for example, what if a field operator could listen to a compressor’s “vital signs” using a stethoscope?

In a related investigation, we considered how a tele-operated robot might carry out tasks considered the domain of a field operator today. We used enactment to sensitize the multidisciplinary team to the variety of deep challenges implicit in telerobotics and the automation of human tasks (Figure 1). In this activity, a remote operator viewed a test rig through a video camera, held by a human “remote inspection robot.” Two blindfolded participants acted as remote robots. The operator gave voice commands they expected the “robots” to understand and the “robots” responded in a way they imagined robots would respond. During this exercise it became clear that deixis was critical, particularly for spatiotemporal references. Physical constraints limit the robots’ activity in ways that the operators cannot fully anticipate—for example, when the operator requested the camera to move right, colliding with a pipe, or the difficulty in communicating the fine movements necessary for manipulating tools. A tangible interface incorporating haptic feedback could be a useful technique for relating the inherent physicality of the guided activity back to the remote operator.

Our prototyping activity was generally oriented toward suggesting potential futures and establishing stakeholder interest in concepts. Taking into account the rigid design requirements and demonstrating a nuanced, grounded perspective on work practices proved critical for concepts to be considered by stakeholders. In a cautious, safety-conscious, and highly regulated industry, technology evolves slowly and operators tend to take a pragmatic, conservative view of innovative technology.


A variety of concepts emerged from and through our design process. Some existed only as paper sketches; others were fleshed out into semifunctional prototypes. These various representations were used within the research project and served as important tools for engaging with external stakeholders and users.

Perimeter+. When work is carried out in a plant, a visual barrier is erected to delineate the work site. The high-visibility barrier helps other operators locate the work site, and if they are passing nearby, alerts them to hazards. The barrier also indicates the intentionality behind the site’s temporal condition. For example, on seeing a strange bypass hose running from a work site, a field operator can reason that it is related to the work being conducted there. Gas sensors are usually worn by field operators, and during some kinds of work activities, bulkier, more sensitive gas detectors are also deployed. In the case of a leak, these portable sensors do not report the alarm back to the control room; the operator must radio the control room. In a crisis situation, this takes extra time, and if the operator is injured, he may not be able to make the transmission at all.

Perimeter+ takes the form of a traditional barrier but contains a number of gas sensors, forming an active safety barrier around the area of highest risk. High-intensity LEDs are used to represent sensor readings, and should a particular threshold be reached, light up in an animated sequence to indicate an escape direction (Figure 2). An alarm signal can then be sent to the plant’s control system over an industrial wireless solution. Because multiple sensors are used and spread over a region, confidence in their reading is improved and response teams are better able to locate the source of the leak.

Flashscope. Facilities typically have thousands of networked sensors reporting values such as pressure, temperature, and flow. Much of this data is unavailable when in the plant, particularly historical trends, which are important for diagnosing faults. Many instruments have small LCD displays for readings, but it is often more convenient for field operators to radio to the control room to find out a reading, even for equipment they are standing in front of.

We wanted to provide operators with more information, yet do so in a way that is empowering and that reflects the importance operators place on the physical environment. Flashscope, a prototype handheld augmented reality system, identifies and tracks the object it is pointed at and projects associated live data back onto the object (Figure 3). The system tracks the object on a plane, adjusting the projection to produce a rough illusion that the visualization is painted onto the object. By using the form of a familiar tool, we hoped that Flashscope could be handled and used in a familiar way and fit into existing practices.

Ambient Awareness. We also approached the problem of information accessibility in the control room and office environment of the plant. Although information is available through computer-based systems, the sheer number of information points, often in the tens of thousands, poses a challenge for bringing up simple overview information in a timely fashion. To support such “glanceable” displays, we developed two simple prototypes. The first uses an off-the-shelf Nabaztag rabbit, which has motorized ears, several color LEDs, and a built-in speaker capable of streaming audio (Figure 4, left). These features were used to indicate the state of an alarm list, with incoming alerts read aloud via speech synthesis. The second concept, dubbed TheatreBox, consists of a sheet of white satin, lit from behind with a color LED strip, and articulated at three points by servomotors (Figure 4, right). Live information is represented through color changes, and historical information is shown by the gradient of the satin. Contextualizing the information is useful, for example, to determine whether a lowering of pressure is part of a longer trend or a sudden change.

Mooves. Control-room operators are at the steering wheels of plants, monitoring and controlling pressure, flow, and temperature with a keyboard and mouse while looking at huge wall displays and two to eight screens per operator. This is mostly dull work at what is essentially a scaled-up office workstation. Control-room operators, however, are experienced field operators, and often take weekly shifts as field operators to keep their knowledge of the physical process fresh. This poses a huge shift in work environment, from the harsh, physical, and tangible environment of the plant to the “flat” interaction with paperwork and office computers.

Mooves is a group of research prototypes based on touch surfaces and physical interaction objects designed to provide efficient and simple interaction with complex automation software for users with varying degrees of computer literacy (see Bridging the gap between the physical work and the highly logical perspective of computer-based control systems, these prototypes provide means for operators to control the plant directly, or to view support systems such as layout diagrams or planning tools. For example, a valve can be controlled by rotating a tangible token on a touchscreen (Figure 5), instead of writing a number on screen, thereby closely linking to the way manual valves are operated by rotating the wheel out in the field. Operators showed interest in the concepts, and some were even more demanding in terms of physicality; they requested force feedback to be included so they could feel when the valve opening was approaching the physical or logical limits.


The oil and gas workplace is quite an unusual setting to design for. The industry is outside most people’s everyday experience, something frequently heard about but seldom experienced firsthand. “Getting a grip” on a setting is an important part of any design process and often raises pragmatic issues, such as the privacy and welfare of participants. The hazardous physical environment of the oil and gas workplace and consequential high demands on safety and security pose novel challenges for the design process. We used techniques such as qualitative fieldwork, enactment, prototyping, and analogous practice to sensitize ourselves to the setting and create new, yet grounded, concepts.

When concepts are demonstrated to industry stakeholders, their early questions are often about pragmatics of deployment. The idea may be sound and the individual unit cost low, but when scaled to the size and complexity of a refinery, it may not offer enough benefit. Considerations of scalability, safety, reliability, and accuracy limit what kinds of technologies can be used, and also severely hamper on-site prototyping and testing. For example, if a wireless mouse is used for controlling the process, measures must be taken to avoid aggravating an already serious situation if the battery runs dry in the middle of a crisis situation.

Given the physical, hands-on nature of the oil and gas workplace, tangible interaction-based designs often seemed most appropriate. Even in the control room, where flat screens and computational resources abound, tangible approaches might better bridge the boundary between working with the physical process and artifacts and the logical process and data. The divide between physical and digital is particularly striking in this workplace, due to the enormity of the physical infrastructure and all the logic and data utilized in making it run. As such, the oil and gas industry, although challenging to work within, offers many interesting challenges for interaction designers.


Most of this work was supported by the Norwegian Research Council and the TAIL IO cooperative research project. We thank the sites that hosted our fieldwork and acknowledge the work of Johan Gunnar, Jon Olav Eikenes, Knut Jørgen Rishaug, Knut Karlsen, and Tommy Lillehagen in the described projects.


1. Heyer, C. High-Octane Work: The oil and gas workplace. Proc. of European Computer-Supported Cooperative Work 2009 (Vienna, Austria). Springer, London, 2009, 363–383.

2. Husøy, K., Graven, T.G., Enkerud, E. Vigilant operators in complex environments: Ethnographic study of oil and gas operations. IFAC HMS Symposium on Analysis, Design, and Evaluation of Human-Machine Systems (Valenciennes, France). 2010.

3. Ljungblad, S. and Heyer, C. Blue-sky and down-to-earth: How analogous practices can support the user-centred design process. Proc. of the 6th Nordic Conference on Human-Computer Interaction (Reykjavik, Iceland). ACM Press, New York, 2010, 723–726.


Clint Heyer is an assistant professor in the Interaction Design Group at IT University of Copenhagen. Research for this article was conducted while he was a postdoctoral fellow at ABB Norway. He is particularly focused on the social dimension of technology and how we design technology that bridges the gaps between established systems, practices, and people.

Kristoffer Husøy is a principal scientist at ABB Technology & Innovation, where he develops user interfaces for automation systems within the oil and gas industry. He has a background in both engineering and social sciences; his research involves developing complex technical systems based on a solid understanding of the context of use.


F1Figure 1. Illustration of enactment exercise with three human “robots” and two remote operators, seated behind them.

F2Figure 2. Perimeter+

F3Figure 3. Flashscope

F4Figure 4. Ambient awareness

F5Figure 5. Mooves

©2012 ACM  1072-5220/12/0700  $10.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2012 ACM, Inc.

Post Comment

No Comments Found