Steve Benford, Adrian Hazzard, Liming Xu
Turlough O'Carolan was the last of the great Irish harpers, an itinerant musician who traveled through Ireland around the turn of the 18th century, staying with and composing tunes for his many patrons. He has also been the inspiration for the Carolan Guitar, a prototype acoustic guitar whose digital augmentations enable it to tell its life story as it passes among players, audiences, and venues.
Our choice of an acoustic guitar to become an example of an Internet-enabled thing is a deliberately provocative one. Acoustic guitars are highly traditional and often valuable, especially when made by hand through the crafting of expensive, delicate tonewoods by a highly skilled luthier. Guitars also tend to be long-lived, existing for decades—sometimes even centuries—and are consequently passed down among owners, quite often outliving them. This makes them an especially challenging target for rapidly evolving and somewhat unstable Internet of Things (IoT) technologies that may well be "here today and gone tomorrow" by comparison. Does it really make sense to digitally augment these venerable instruments?
And yet it is the inherent value and longevity of these guitars that also makes them an ideal target for the IoT. Guitars gather rich stories throughout their lifetimes that enhance their provenance and value. From the sustainable sourcing of the tonewoods used to construct them, through the processes of construction, to the history of their ownership, to the tunes they have played, guitars are replete with stories. These stories can add great value to guitars, with "celebrity" instruments being sold for millions of dollars. Mind you, everyday guitars may also be extremely valuable to their owners, being associated with very unique personal memories and sometimes being handed down from generation to generation.
So we are all agreed that we should augment acoustic guitars with the ability to capture and tell their rich life stories. The next question is, how might we do this? Again, we have opted for what might prove to be a provocative approach. We rejected the idea of embedding sensors, actuators, and displays within these objects (even though there is plenty of room inside them) in favor of covering them with interactive surface decorations. Part of the craft of luthiery is to decorate the instrument with beautiful inlay, typically on the headstock and in the rosette around the soundhole, but sometimes extending to the bindings and on occasion to other places too.
We have chosen to work with a technology called Aestheticodes  that enables people to draw their own interactive patterns from scratch. By following a simple set of topological drawing rules that concern the numbers and arrangements of solid "blobs" and "regions" embedded within a drawing, designers are able to create beautiful patterns that may contain multiple points of interaction hidden within them. Viewers can then locate these codes within a wider pattern and scan them with their phone or tablet to trigger digital interactions such as viewing some media, visiting a website, or leaving a comment. This topological approach was previously explored by Enrico Constanza and implemented in a system called D-Touch . We have since built on his pioneering work to extend the technique and implement it within a mobile app that you download for free (www.aestheticodes.com).
Our initial experience of working with skilled designers to create interactive bowls, menus, and placemats for a restaurant revealed that the relative simplicity and openness of the "recognition rules" matched well with their valued drawing skills and enabled them to create rich interactive patterns . We therefore were determined to apply the same approach to decorating a handmade acoustic guitar.
We brought together an interdisciplinary team comprising a luthier, a graphic designer with experience with Aestheticodes, and HCI researchers. This team set about designing and building a guitar over a period of six months. We are documenting our progress on the guitar's blog (www.carolanguitar.com), including reports of where it has been and who has played it since it was released into the wild. We'll quickly summarize a few key points here.
Inspired by the O'Carolan theme, our designer began by creating some Celtic knotwork Aestheticode patterns. Our luthier then took us through the process of choosing the guitar's woods, shape, and size. We elected for a combination of spruce for the soundboard and flamed maple for the back and sides, and chose a basic dreadnought shape for the body, subtly extending some of the bouts to accommodate more extensive patterns. Perhaps the most interesting aspect of the design—at least for this readership—lay in deciding where on the guitar to place the interactive patterns and which interaction each might trigger. While this mapping can, and no doubt will, evolve over time, several ideas emerged from our early discussions:
- The headstock would carry a logo that would link to official provenance.
- The public areas on the front soundboard would link to videos of performances with the guitar.
- The largest area of real estate on the instrument, the back, would be decorated with a large code that might be scanned from some distance so audiences could learn about its current player as well as the history of previous players.
- The areas under the strings would be reserved for patterns that linked to technical documentation describing construction and maintenance that only someone who removed the strings would be able to access.
By far the most complicated part of the design was the soundboard, as this involved adapting the patterns to meet several constraints, which ultimately led us to some unusual construction techniques. The soundboard is an especially sensitive part of the guitar that contributes greatly to its voice. It is made from an expensive and thinly carved piece of wood that is placed under great tension by the strings so it must be supported by underlying braces, the positions of which are part of the luthier's art. The soundboard must also have holes, with sufficient total area to project the instrument's voice, but not in the sensitive area of the lower bout, as this would compromise structural integrity. Finally, the player's arms tend to obscure parts of the soundboard.
The resulting pattern was carefully designed to meet all of these constraints while also containing two distinct interactive codes, one on the areas visible to the public and a second one running under the strings. This pattern was realized using a combination of laser etching to mark out the pattern and laser cutting to make the soundholes and also cut out the inlay from darker rosewood and mahogany, with the inlay then being applied by hand. The codes in the final design comprise a mixture of inlay and holes (which appear dark) to produce a pattern that is scannable but that also functions as a soundhole.
Since its completion, our guitar has found its way into the wider world and into the hands of players who have told us their stories of guitars and their thoughts about the project, and who have recorded tunes and songs [4,5].
While it's lovely to talk in detail about building guitars, we should also consider what this project might tell us about the IoT. We suggest there are three broad possibilities here.
First, constructing the Carolan guitar has already revealed something about the complex nature of digitally augmenting physical artifacts. Applying interactive surface decoration has required us to become intimately familiar with the material properties arising from the quality and nature of wood and of techniques for etching, cutting, and inlaying it, mirroring the observations of Tsaknak, Fernaeus, and Schaub on their experience of crafting digital interactions into leather . Crafting interactivity into a complex real-world artifact has also required us to take account of its structural qualities, including understanding good and bad places to apply patterns as well as functional properties, such as how its voice is produced, how players will hold it, and even how one removes its strings. In short, there would appear to be far more to augmenting a thing than "slapping" a QR code somewhere onto its surface (or electronics into it).
As we move forward into the next phase of the project, we hope the Carolan will help us answer two further questions with implications for the wider IoT.
One key question concerns how to augment everyday objects in a sustainable way. Guitars can have long lifetimes and typically evolve slowly in comparison with digital technologies. How can we mitigate the challenges of our digital technologies becoming obsolete long before our instrument reaches the end of its active life? Is it more sustainable to integrate electronics into a traditional artifact, or to connect it to digital media through interactive surface decoration, as in the case of the Carolan?
Another major question concerns the kinds of stories that might become associated with our guitar and the various ways in which these might enhance its value. Previous IoT research has begun to explore how stories can enhance the value of everyday things . We intend for the Carolan to help us understand the wide range of stories that can be associated with an object from the moment of its conception and birth, through its ongoing use as it passes among many different owners, to its ultimate archiving. Even at this early stage, we have seen plenty of evidence that our guitar will acquire a rich digital footprint encompassing the details of its construction, videos of performances, and the wider stories and reflections it elicits from players.
We look forward to being able to answer these questions as our project progresses. We hope you will follow our progress at www.carolan.com and perhaps even get to meet and play the Carolan as it continues its journey.
1. An introduction to Aestheticodes; https://www.youtube.com/watch?v=kW39Mt5kscQ
3. Meese, R., Ali, S., Thorne, E.C., Benford, S., Quinn, A., Mortier, R., Koleva, B., Pridmore, T., and Baurley, S. From codes to patterns: Designing interactive decoration for tableware. Proc. CHI 2013. ACM, 2013.
4. The Carolan guitar being played by Gypsy Jazz guitarist Lulo Reinhardt; https://www.youtube.com/watch?v=M6ctCMUwx70.
5. Lulo Reinhardt recounts his experience of decorating guitars; https://www.youtube.com/watch?v=j6IGA4A34WA
Steve Benford is professor of collaborative computing at the University of Nottingham's Mixed Reality Laboratory and director of the Horizon Centre for Doctoral Training. His research focuses on cultural applications of computing systems. He is a keen amateur guitarist. firstname.lastname@example.org
Adrian Hazzard is a research associate at the University of Nottingham's Mixed Reality Laboratory. His research focuses on music in HCI, primarily on adaptive musical experiences. He is also a musician and composer. email@example.com
Liming Xu is a Ph.D. student at the International Doctoral Innovation Centre at the University of Nottingham, where he is researching interactive vision technologies, including the development of Aestheticodes. firstname.lastname@example.org
©2015 ACM 1072-5220/15/05 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.