sketches
text archive
equipment layout
Projektbeschreibung
East by West

a networked environment

 

Johannes Birringer, Sher Doruff, Orm Finnendahl

Johannes Birringer
Environments Lab
1813 N High Street
Columbus, OH 43210 USA
ph/fax: +1 614 299 0954
birringer.1@osu.edu
Sher Doruff
Waag Society f. Old&New Media
Nieuwmarkt 4
1012 CR Amsterdam
Tel.: +31 20-5579898
sher@waag.org
Orm Finnendahl
Seelingstr. 47/49
D – 14059 Berlin
Tel.: +49 30-3412405
finnendahl@folkwang-hochschule.de
http://www.folkwang.uni-essen.de/~finnendahl

concept & artistic production:
Johannes Birringer (USA/Germany)
Sher Doruff (Holland)
Orm Finnendahl (Germany)

production assistance:
Ayse Orhorn (Turkey)
Sibylle Uttikal (Germany)
Motion tracking design: Frieder Weiß (Germany)
programming and systems:
Software: PD sound design (Finnendahl), GNU/Linux: Miller Puckette (PD), Paul Davis (RME driver), Bill Schottstaedt (sndlib) and Stefan Kersten (adoption of PD to sndlib), Heiko Schlittermann (LUG Dresden); EyeCon (Weiß); Keystroke (Doruff, and Waag Society, Tom Demeyer, Niels Bogaards, Just van den Broecke).

 

press review


 

0.1. Description


"East by West" consists of two interactive and distributed environments constructed at opposite ends of a building and connected via live video-audio streaming. Each environment creates a landscape: the east is a garden where hundreds of oranges hang suspended from the ceiling; the west is a sandy beach with numerous boccia balls. Both landscapes invite the visitor to explore and play with the objects in the environments.

Although the title of the installation alludes to a Hitchcock thriller, the geographical references are here site-specific, namely to the East and West Studios of the Hellerau Festspielhaus and its grand symmetric architecture which was built in 1911. The twofold design for the environment -- first implemented during the production workshop "Realtime & Presence" organized by the Trans-Media-Academy Hellerau in July 2002 -- intends to generate a playful dialogue between the two rooms involving real-time sound and visual synthesis (telepresence) responsive to the behavior of the visitors.

[At CYNETart2002 in November 2002 we present a new version of the design process which explores the emergence and temporal synthesis of musical, visual and kinaesthetic perceptions in two similar yet different "geographic" architectures. The synthesis underscores the experience of the visitors and their strategic play or intuitive interaction with potential games or performance environments (the boccia strand, the orange garden). In subsequent phases of the project, a single local installation-site will be linked with remote sites in other countries and with other collaborators in Europe, North America, Japan and Brazil.]

Our objective is to experiment with the transformation of the spatial imaginary (real space as virtual space) and the experience of time and simultaneity in complex, dynamic systems. The experience is generated through sound and actions/behaviors of the visitors in environments of hyperplasticity.

The term "hyperplasticity" refers to the emergent relationships between visitors in both sites as they engage with the spaces and the "transobjects" they find in the spaces. The "transobjects" are partly real (boccia balls, sand, oranges, fabrics), but they are also constituted and reflected as visual and aural transmissions (streaming video/audio images) which flow back and forth between the two sites in real time. In addition, laptop keyboards in each site allow visitors to send text messages to each other across the distance; the live messages scroll across the projected telepresence images. The fluctuating conditions in both environments will depend on the behavior of the visitors. The social and aesthetic dimension of the work therefore depends on a careful examination of interactivity understood here as process through which meanings of locales and milieus are constantly evolving, adaptable and redefinable.

The political or geographical markers, "East" and "West," consequently lose their initial associations, but as the design process continues, the installations will increasingly focus on the communications (tele-actions) between players in remote sites -- different cultural and linguistic locales which may generate different "rules" of behavior. In an ecological sense, the visitors will need to adapt to their shared environments. In version 2.0 of "East by West," the visitors’ activity in the west is tracked by Eyecon cameras, and graphic representations
of the spatialized sound are projected onto the sand, giving the audience a plastic opportunity to "walk," dance or play on the sound-spheres while they can also see the interaction of the boccia and orange games projected in the telepresence window (on one side of the room). The new version will use graphic projections of sound-space in the west room; subsequent installations will use them in both rooms.

Finally, the concept of networked, translocal spaces allows investigation of the nature of real-time sound synthesis and how extended physical space can be shared by people when they play with fictional geographies, strange or familiar objects, and their mediated presences. Telepresence restructures and enlarges the environment with its projection (window) of mediated and combined presences in action. The window is transducive: telepresence effects physical action, physical action affects the symbiotic movement within the networked stream, and thus also the ways in which the players can interpret the gestures of their performance. Linking a "local" site with a "remote" site raises particular challenges for our understanding of new artistic paradigms in telepresence, distributed and "navigational" art. Beyond conventional
forms of performance and video art, distributed communication in real time requires the recipient to be active. The social orientation toward sensual environments and "hyperplasticity" is not directed at euphoric assumptions about virtual reality (VR) but at concrete, synaesthetic processes of cognition. "East by West" addresses the visitors' playful fantasy and tactile exploration of the environment; the interface becomes useful if such play recognizes how parallel reality-systems can converge or affect each other, how we integrate other realities into our social experience.

 

0.1.2. Installation Prototypes: Description of Acoustic Space, Interactive Streaming Architecture and “Transobjects”

 

The project combines perspectives and artistic methods which each of the artists involved brings to the collaboration from their different backgrounds. The following section sketches the approaches we have taken to the joint project.

 

Point of departure for collaboration:
(Interactive Sound, Orm Finnendahl)


Our project proposes research into interaction of movement/sound/space and movement/visuals with reference to some properties of Finnendahl’s multichannel acoustic pieces that are focused on transformation in realtime.

On the level of sound, our research means conducting various tests:


- how do different sounds work, when they are placed in different illusionary spaces around someone in reaction to his/her movement or behavior in the rooms?


- what kind of experience is it, when there are visual objects and video projections around the person defining an optical space which gets transformed (prolonged, narrowed, widened, altered) according to some movement/behavior of this person?


- what happens, if sound and visuals get interconnected (e.g, the acoustical space moves, while the visual widens...)? How does the visitor experience the interaction that changes the meaning of the visual objects?


During the Hellerau workshop, we build the two environments and install the sound programming first, starting with an archive of sounds and its transformations. We will explore how the same sound objects are experienced differently in different environments, and how different environments and communicative situations change our perception of sound

 

.Point of departure for Collaboration:
(Interactive-Streaming Video, Sher Doruff)


The idea of creating acoustic space(s) with temporal shifts elicited Doruff’s interest in the convergence of perceptual nuance in physical space that may (or may not) be somewhat changed by another space. That space might be a 'middle space' (virtual space) or it may be another remote physical location(s). In the collaboration we are interested in transmitting the qualities of spatialization from one site to another space in some way.


Our concept for a “middle space” or virtual environment is based on the notion of acoustic-virtual spaces and image-spaces that are experienceable in more than one location since these events are transmitted, thus always unfolding.


The unfolding of fluctuating conditions is an architecture of time with temporal shifts, delays, simultaneities, returns, emergences, a mixing of spatial experiences where the virtual is not becoming real but already real yet different, a passage from one state to another, a constant differentiation, dynamic, uncertain..... The virtual does not have to be realized, it is already somewhere else.


Along with the sound programming, we will install web-cameras and projectors for the real-time telepresence interaction using Keystroke, a Multi-User Cross Media Synthesizer.
Keystroke is a distributed application that allows multiple players to generate, synthesize and process images, sounds and text within a shared realtime environment. As an instrument (used in both East and West spaces), it will allow programmers and visitors, present in both rooms at the same time, to dynamically affect various aspects of the virtual environment (visual projections, imaginary space) - the flows and exchanges between the sites

 

Point of departure for Collaboration:
(Telepresence Performance/Hyperplastic Design, Johannes Birringer)

Conctructing interactive space implies questions about the nature of “installation design” in the context of shared realtime environments that are networked/distributed. Birringer’s interest lies in movement experienced through images, voice, abstract sound as well as small narrative modules or stories or "situations" that arise from actions. Such stories may or may not involve the body, but it is of crucial interest to investigate networked conditions, how they are experienced and used by collective subjects, how physical consciousness is reshaped in digital or "virtual" environments, how people connect into these environments kinesthetically and psychologically over time.


Performative questions for Interactive / Telepresent Space therefore have much less to do with choreography or preformed vocabularies; rather, they address cognitive processes:


-- how do we today navigate - orient ourselves - in the world (in mediated space, networked space)?


- what are our means of information from which we infer where we are ('positioning system') and how we move - especially if space - as networked condition of existence, or interdependence - is reliant on "objects" of information gathering/locationing (cell phones, internet, maps, interactive scenarios in everyday life, object-relations, sensors, markers, orientation signs, sound information)?


- how do we cope with contingent, indeterminate, open, fluid "locations" (when the meaningfulness of a location changes)?

 

 

* * *

 

For the “architecture” of our installation design, we initially defined an approach that places emphasis on abstract acoustic space being self-referential as such. Our developed version of this approach is more sculptural, referential, and sociological and involves the idea of the “transobjects.

Testphase: East by West - illusionary spaces of sound

In an initial approach, we concentrated on a setup where visitors would be interested in making experiences with the space-transforming sound on their own. Both rooms are empty, the walls functioning as projection surfaces. We prepare the most effective visuals/acoustic materials and means of transformation.

The focus of this idea is not so much content oriented, but more situation oriented: What would it be, if there are 2 persons, each in one (similar/identical) room. If one person moves (bends down, waves his arms, walks around...), the acoustic space widens/narrows. But the visitor not only controls his/her own acoustic space, but also the visual space in the other room. If the same is done in the other room, but crosslinked, a communication between the two evolves. This is not a spoken word communication, but a mediated communication: the person moves in a certain direction to change his/her acoustic space (e.g. in order to make it congruent with the visual perception), and at the same time the person triggers some reaction of the person in the other room, who tries to do the same from his/her perspective (maybe both movements contradict each other, leading to some frustration on both sides, as the actions, which are taken, lead to the opposite results of their intentions due to its feedback structure). Although a seemingliy formalistic approach this could well get interpreted in a political or social way without having to refer to concrete images/sounds.

 

Prototype: East by West - translocal situations


In the extended approach, we imagine constructing two environments which are not identical but compose a different metaphorical ambience or situation inviting the visitor to interact with textures and tangible objects in the environment.

These environments create changing imaginary space through the sound transformations, while the telepresence connection between East and West rooms implies that the visual images of more or less identifyable persons in these rooms get the visitors socially and perceptionally involved as there are some direct affective relation between them and the projected images. The visual spaces are transformed in their physical dimensions as well (which is central to what Finnendahl is interested in musically), since the real-time transmissions from the other room extend the single room, and some objects may begin to function as “trans-local props” (e.g. the sand, boccia balls, oranges). Using the real-time cross-media synthesizer allows for a range of possible manipulation and distortion of the transmitted images, thus exploring further the transformations of spatial realities though sound and visuals, real-time trompe l'oeil, extensions and translocations of the room-realities.

 

Diagrams


IThe diagrams show the West room as a “play-space” with a downward gravitational pull towards the sand-boxes and the game of bocchia to which the visitors are invited. The West room is earth-centered and has dim warm lighting, and at the same time might evoke explorations of the bocchia game or sandbox playfulness in the sense in which such a game “geography” involves imaginary rules.


The East room is a more abstract sculptural installation that is aerial, showing eighty or ninety pieces of oranges hanging in the space suspended from the ceiling and lit with spotlights from above. A broken door also hangs under the ceiling. At first sight, the room might create a surreal feeling, but also invite the visitor to explore the possible functions or evocations of the suspended oranges.


As the visitor moves around, the sound in the space changes, and visual projections will appear on both sides of the walls in each room. The telepresence "windows" mix the presences in each room. The projection design is under development; in later versions of this installation we add floor projections of graphic representations of the changing souund, activated by Eyecon tracking cameras that respond to movement and control some of motion of the image projections and the lighting in the space. Furthermore, the hanging objects will become more kinetic, their behavior more unpredictable as their supended movement (up and down) on pulleys -- activated telerobotically -- will be affected by actions of the visitors in the West room. Thus, a kind of surreal ballet méchanique, a Bunraku-ballet of fruits might occur in the East room.



In terms of a political or territorial imaginary, the two rooms will be completely abstract. The underlying narratives that we wrote -- for the sandy beach/dead sea and the "Orange County" -- may never become known to the visitors. We anticipate, however, that the real-time presence of the acoustic and visual flows, inflections, loops, and alterations will generate certain recursive structures and newly evolving spatial relations that may be stimulating and thought-provoking. The real-time synthesis, performed together by the visitors, the programmers and the controllers that activate some of the transmissions, will generate unpredictable variations, in the constant mix and remix of the plastic, acoustic, and projected dimensions of the continuum between the two sites.

 

 

 

0.2. Participants


Johannes Birringer (USA/Germany)
Sher Doruff (The Netherlands)
Orm Finnendahl (Germany)


production assistance
Ayse Orhon (Istanbul), Sibylle Uttikal (Berlin)


Technical assistance:
Niels Bogaards (WAAG, Amsterdam)
, Frieder Weiss (Nuremberg), Udo Zickwolf (Dresden), Thomas Dumke (Dresden), Kai and Ulf (Dresden)

 

0.3. Technical plan/equipment

Audio:


2 computers with multichannel io cards and MIDI/network

1 computer for the mapping of control data
i2 Audio Mixers
2 Amplifier
16 Loudspeakers


Video/telepresence:


Space East -


1 - G4's min 256ram
2 - video projectors (or more plus vga splitter)
vga cables
audio cables
1- usb audio interface
1 - midi interface
1 - iMac as server
1 -iMac, laptop or G4 as KS client
2- webcams
1 dv camera


Space West


1 - G4's min 256ram
2 - video projectors (or more plus vga splitter)
vga cables
audio cables
1- usb audio interface
1 - midi interface
1 -iMac, laptop or G4 as KS client
2- webcams
1 dv camera


Network


ethernet cables + hi-bandwidth connection
5 -static ip's on a subnet
the server can sit in either room or outside the rooms. same is true
for the KS clients, depending on the interaction that's chosen.
positions of cameras not yet decided

Lighting instruments


12 lighting instruments (Leico and/or Fresnell)
12 50 or 100 ft extension cords
dimmers

Construction materials


for the installation of two environments (lumber, glass, steel, wires, sand, fabrics, coathangers, etc)

 

0.4. Budget items

Construction materials for the installation of two environments
(lumber, wires, sand, objects, etc)


lights (provided by host organization)


tapes, electronic components, materials


video projectors
webcams
audio equipment

(availability or rental of equipment need to be clarified with host organization)

artistic fees
(for three principal investigating collaborators)
Additional fee for other collaborators/programmers


travel
(

TOTAL (projecetd) 18,000.00

 

0.5. Installation diagrams

See diagrams on our website

 

0.5.0 Diagram of empty space

0.5.1 Diagram of West Room

0.5.2 Diagram of East Room



(diagrams are under development and subject to modification)

 

04.20.02

 


NavMenu
Core Page Schedule Gateway News Gallery People Projects