Remnance of form

Issue: XXII.5 September-October 2015
Page: 12
Digital Citation

Authors:
Sang-won Leigh, Asta Roseway, Ann Paradiso

Describe what you made. Remnance of Form (typo intentional) is an interactive installation that explores the dynamic tension between an object and its shadow. The computer-generated shadow can transform into any shape, creating a new narrative that challenges our perception of reality.

Our setup allows viewers to directly manipulate the lightbulb. As the lightbulb starts to swing like a pendulum, the computer-generated shadow becomes much more realistic because our mind automatically picks up the synchronized motions of the lightbulb and the shadow. This, combined with the viewers' presences and body postures, allows the interaction between the viewers and the object (a white ball) with its shadow to become more playful and perceptually appealing.

Briefly describe the process of how this was made. The key is to understand and leverage the geometric relation among light, object, and shadow for making a believable experience.

First, a realistic computer-generated shadow can only be created through front projection, because back-projection generates less natural shadow effects due to different light scattering and diffusion. For front projection, a projector had to hang high above to avoid the object (the ball) getting in the way of the projection. The lightbulb hangs behind where the projector is for the same reason.

The second step is to remove visual interferences as much as possible. A shiny acrylic sheet on the floor is used to cancel out natural shadows. The brightness of the projection and the bulb are carefully optimized so that any remaining natural shadows are overridden.

Last, the cameras (IR camera and Kinect) are installed. The ball is covered with reflective paint for reliable IR tracking; the IR camera tracks the lightbulb as well. The Kinect sensor tracks viewers' positions and gestures.

What for you is the most important/interesting thing about what you made? Shadows are ambient and sophisticated—they are hard to manipulate or remove, but also are very subtly manifested and hidden in the corner of our attention. Due to this subtlety and complexity, viewers sometimes don't even recognize the alteration of shadows, later haphazardly discovering it with surprise.

What expertise (skills and competences) did it require? Computer vision and a basic understanding of lighting.

What materials and tools did you use? Physical components include the materials (globe, acrylic sheets, foam boards, lightbulb, etc.) used for creating the canvas for shadow interactions. Digital components (projector, cameras, PC) enable the interactions by augmenting the canvas with computer-generated shadows. We used openFrameworks for the software implementation.

What was the biggest surprise in making this? The use of the hanging lightbulb kit was decided at a very late moment in the creation process. At the time it was a purely aesthetic decision; however, it turned out that manipulating the lightbulb provides playfulness and creates a strong illusion that the shadow is real.

What will you repeat in another project that you did well in this project?

We think the poetic communication through shadows can go beyond the form of a staged installation. It would be interesting to re-implement the shadow augmentation on much larger scales or with different objects, either for an artistic or more functional purpose such as digital notifications.

Sang-won Leigh, MIT Media Lab
[email protected]

Asta Roseway, Microsoft Research
[email protected]

Ann Paradiso, Microsoft Research
[email protected]

http://www.sangww.net/2015/02/remnance-of-form.html

Figures

UF1 Figure. Early explorations on shadow augmentation.