Digital Culture & AudioVisual Challenges

In this paper, we examine the conjunction between (1) the mythologic concepts of auditory echo and reflection, (2) their extension to visible phenomenon, (3) a new digital technology and a new interactive art.

We introduce a new concept, called “interactive simulated matter”allowing to render wide variety of echoes, reflections or more or less permanent absorbing effects in response of actions. We present 3 artistic creations experiencing this concept with a software called GravDyn, for Dynamic Engraving, developed by ACROE.

1. Introduction
In this paper, we examine the conjunction between (1) the mythologic concepts of auditory echo and reflection, (2) their extension to visible phenomenon, (3) a new digital technology and a new interactive art. We introduce a new concept, called “interactive simulated matter”allowing to render wide variety of echoes, reflections or more or less permanent absorbing effects in response of actions. We present 3 artistic creations experiencing this concept with a software called GravDyn, for Dynamic Engraving, developed by ACROE.

2. Presentation of the concept
In Greek mythology, Echo is a nymph punished by Hera who can only repeat the last words she hears. In its usual sense, echo is what is heard by a listener, facing a petrified world, which hears the last words spoken. In both cases it is a phenomenon which manifests itself in the acoustic dimension. The myth of Narcisse, mirroring himself in the water is a similar kind of phenomenon, but transposed optical phenomenon.

However, if we consider, not the point of view of the nymph, of the listener or of the viewer, but that of the world they are facing with, such perceptual phenomena are perceptual expressions of the interaction between someone and the world. Thus, by shifting from both acoustic and optical situations to a more global interaction with the world, we generalize the echo phenomenon as a kind of response of the world to our actions. Taking the case of a petrified world (a mountain for example), “echo”reflects - mimics - emitted sounds.

Suppose we can simulate this world through physical computer simulations, we will be able to generate not only acoustical echoes, but several other types of how this world reflects our actions. The world’s feedback could be acoustic but also visual and gestural and more generally multisensory. More generally, we could generalize“echoes”or reflections, not only to“imitate”our actions but to transform them.

We have called this new point of view: “real gestures on simulated matter”and we will experience how it opens up new avenues for the concepts of echo, propagations, absorbtions, etc.

3. Scientific and technical developments
We developed a software called“Dynamic Engraving”. It consists on the modeling and simulating a physical surface that exhibits dynamic behaviors such as programmable reflection, propagation, engraving, according to its physical parameters. The parameters can be changed allowing to create various degrees of such phenomena: from perfect echo and reflection, to perfect absorbsion as more or less permanent engraving. We interact gesturally with such physically-based surface, and so doing, the react of the surface is such as a dynamic answer to our actions.

4. Artistic Works
We experienced such concept and technics in three artistic visual, musical and poetic creations:
•“EtOndes..Particules”(Image 1), designed and created by Kevin Sillam, scientific PhD researcher during his doctoral internship and Melanie Lesbats, young artist during her internship, working at ACROE with Annie Luciani.
•“Somnambule”(Image 2), designed and created by Kevin Sillam, scientific PhD researcher during his doctoral internship and Noé Guirand, young composer during his studies at Music Conservatory in Grenoble.
•“Wanderings”(Image 3), created and designed by Annie Luciani, who designed the software “Dynamic Engraving”.

In“EtOndes..Particules”, we interact with a simulated physical surface, causing propagations playing like an echo of the movement of his body, felt by a camera. The simulated physical surface reacts to the body gestures by creating multiple and varied dynamic echoes more or less transformed. The process can be collective: several people can interact simultaneously with the same simulated world. Behaviors such as multiple echoes of one action can emerge generating multiple ghost echoes.

In ”Somnambule”, a pianist interacts through his musical gesture with a physically-based puppet simulation. The puppet dynamically engraves the simulated physical surface, creating dynamic traces as in more or less fluid water. The dynamic trace of the puppet also controls another - its ghosts - similar but different from the first one, also engraving the same simulated material. Such a cascade of dynamic traces of the pianistic gesture aims to represent a dream of dreams as successive echoes of an initial gesture.

In “Wanderings”, the simulated physically-based surface presents several zones, with physical parameters just a little bit different. Thus, at each frontier, the fluid propagation is more or less disrupted. By managing the physical parameters on the frontiers, echo and propagation are able to cross more or less the frontier, from total echo refection to waves propagation, and between both cases, we obtain a large panel transformed echoes, propagations, ephemeral or remanent absorbsions, etc.

5. Conclusion
Such simulated phenomena enlarge the realistic ones - acoustical echo or optical mirroring - by creating intermediate situations of propagation, echoes and permanent or ephemeral absorbsions and engraving. The concept of interactive and dynamic simulated matter illustrates the concept of heterotopia with an intimate intertwining of space and time.