Johannes
Birringer, Sher Doruff, Orm Finnendahl
Johannes
Birringer
Environments Lab
1813 N High Street
Columbus, OH 43210 USA
ph/fax: +1 614 299 0954
birringer.1@osu.edu |
Sher Doruff
Waag Society f. Old&New Media
Nieuwmarkt 4
1012 CR Amsterdam
Tel.: +31 20-5579898
sher@waag.org |
Orm Finnendahl
Seelingstr. 47/49
D 14059 Berlin
Tel.: +49 30-3412405
finnendahl@folkwang-hochschule.de
http://www.folkwang.uni-essen.de/~finnendahl |
concept
& artistic production:
Johannes Birringer (USA/Germany)
Sher Doruff (Holland)
Orm Finnendahl (Germany)
production assistance:
Ayse Orhorn (Turkey)
Sibylle Uttikal (Germany)
Motion tracking design: Frieder Weiß (Germany)
programming and systems:
Software: PD sound design (Finnendahl), GNU/Linux: Miller Puckette
(PD), Paul Davis (RME driver), Bill Schottstaedt (sndlib) and Stefan
Kersten (adoption of PD to sndlib), Heiko Schlittermann (LUG Dresden);
EyeCon (Weiß); Keystroke (Doruff, and Waag Society, Tom Demeyer,
Niels Bogaards, Just van den Broecke).
press
review
0.1. Description
"East by West" consists of two interactive and distributed
environments constructed at opposite ends of a building and connected
via live video-audio streaming. Each environment creates a landscape:
the east is a garden where hundreds of oranges hang suspended from
the ceiling; the west is a sandy beach with numerous boccia balls.
Both landscapes invite the visitor to explore and play with the objects
in the environments.
Although the title of the installation alludes to a Hitchcock thriller,
the geographical references are here site-specific, namely to the
East and West Studios of the Hellerau Festspielhaus and its grand
symmetric architecture which was built in 1911. The twofold design
for the environment -- first implemented during the production workshop
"Realtime & Presence" organized by the Trans-Media-Academy
Hellerau in July 2002 -- intends to generate a playful dialogue between
the two rooms involving real-time sound and visual synthesis (telepresence)
responsive to the behavior of the visitors.
[At CYNETart2002 in November 2002 we present a new version of the
design process which explores the emergence and temporal synthesis
of musical, visual and kinaesthetic perceptions in two similar yet
different "geographic" architectures. The synthesis underscores
the experience of the visitors and their strategic play or intuitive
interaction with potential games or performance environments (the
boccia strand, the orange garden). In subsequent phases of the project,
a single local installation-site will be linked with remote sites
in other countries and with other collaborators in Europe, North America,
Japan and Brazil.]
Our objective is to experiment with the transformation of the spatial
imaginary (real space as virtual space) and the experience of time
and simultaneity in complex, dynamic systems. The experience is generated
through sound and actions/behaviors of the visitors in environments
of hyperplasticity.
The term "hyperplasticity" refers to the emergent relationships
between visitors in both sites as they engage with the spaces and
the "transobjects" they find in the spaces. The "transobjects"
are partly real (boccia balls, sand, oranges, fabrics), but they are
also constituted and reflected as visual and aural transmissions (streaming
video/audio images) which flow back and forth between the two sites
in real time. In addition, laptop keyboards in each site allow visitors
to send text messages to each other across the distance; the live
messages scroll across the projected telepresence images. The fluctuating
conditions in both environments will depend on the behavior of the
visitors. The social and aesthetic dimension of the work therefore
depends on a careful examination of interactivity understood here
as process through which meanings of locales and milieus are constantly
evolving, adaptable and redefinable.
The political or geographical markers, "East" and "West,"
consequently lose their initial associations, but as the design process
continues, the installations will increasingly focus on the communications
(tele-actions) between players in remote sites -- different cultural
and linguistic locales which may generate different "rules"
of behavior. In an ecological sense, the visitors will need to adapt
to their shared environments. In version 2.0 of "East by West,"
the visitors activity in the west is tracked by Eyecon cameras,
and graphic representations
of the spatialized sound are projected onto the sand, giving the audience
a plastic opportunity to "walk," dance or play on the sound-spheres
while they can also see the interaction of the boccia and orange games
projected in the telepresence window (on one side of the room). The
new version will use graphic projections of sound-space in the west
room; subsequent installations will use them in both rooms.
Finally, the concept of networked, translocal spaces allows investigation
of the nature of real-time sound synthesis and how extended physical
space can be shared by people when they play with fictional geographies,
strange or familiar objects, and their mediated presences. Telepresence
restructures and enlarges the environment with its projection (window)
of mediated and combined presences in action. The window is transducive:
telepresence effects physical action, physical action affects the
symbiotic movement within the networked stream, and thus also the
ways in which the players can interpret the gestures of their performance.
Linking a "local" site with a "remote" site raises
particular challenges for our understanding of new artistic paradigms
in telepresence, distributed and "navigational" art. Beyond
conventional
forms of performance and video art, distributed communication in real
time requires the recipient to be active. The social orientation toward
sensual environments and "hyperplasticity" is not directed
at euphoric assumptions about virtual reality (VR) but at concrete,
synaesthetic processes of cognition. "East by West" addresses
the visitors' playful fantasy and tactile exploration of the environment;
the interface becomes useful if such play recognizes how parallel
reality-systems can converge or affect each other, how we integrate
other realities into our social experience.
0.1.2.
Installation Prototypes: Description of Acoustic Space, Interactive
Streaming Architecture and Transobjects
The project combines perspectives
and artistic methods which each of the artists involved brings to
the collaboration from their different backgrounds. The following
section sketches the approaches we have taken to the joint project.
Point
of departure for collaboration:
(Interactive Sound, Orm Finnendahl)
Our project proposes research into interaction of movement/sound/space
and movement/visuals with reference to some properties of Finnendahls
multichannel acoustic pieces that are focused on transformation in
realtime.
On the level of sound, our research means conducting various tests:
- how do different sounds work, when they are placed in different
illusionary spaces around someone in reaction to his/her movement
or behavior in the rooms?
- what kind of experience is it, when there are visual objects and
video projections around the person defining an optical space which
gets transformed (prolonged, narrowed, widened, altered) according
to some movement/behavior of this person?
- what happens, if sound and visuals get interconnected (e.g, the
acoustical space moves, while the visual widens...)? How does the
visitor experience the interaction that changes the meaning of the
visual objects?
During the Hellerau workshop, we build the two environments and install
the sound programming first, starting with an archive of sounds and
its transformations. We will explore how the same sound objects are
experienced differently in different environments, and how different
environments and communicative situations change our perception of
sound
.Point
of departure for Collaboration:
(Interactive-Streaming Video, Sher Doruff)
The idea of creating acoustic space(s) with temporal shifts elicited
Doruffs interest in the convergence of perceptual nuance in
physical space that may (or may not) be somewhat changed by another
space. That space might be a 'middle space' (virtual space) or it
may be another remote physical location(s). In the collaboration we
are interested in transmitting the qualities of spatialization from
one site to another space in some way.
Our concept for a middle space or virtual environment
is based on the notion of acoustic-virtual spaces and image-spaces
that are experienceable in more than one location since these events
are transmitted, thus always unfolding.
The unfolding of fluctuating conditions is an architecture of time
with temporal shifts, delays, simultaneities, returns, emergences,
a mixing of spatial experiences where the virtual is not becoming
real but already real yet different, a passage from one state to another,
a constant differentiation, dynamic, uncertain..... The virtual does
not have to be realized, it is already somewhere else.
Along with the sound programming, we will install web-cameras and
projectors for the real-time telepresence interaction using Keystroke,
a Multi-User Cross Media Synthesizer.
Keystroke is a distributed application that allows multiple players
to generate, synthesize and process images, sounds and text within
a shared realtime environment. As an instrument (used in both East
and West spaces), it will allow programmers and visitors, present
in both rooms at the same time, to dynamically affect various aspects
of the virtual environment (visual projections, imaginary space) -
the flows and exchanges between the sites
Point
of departure for Collaboration:
(Telepresence Performance/Hyperplastic Design, Johannes Birringer)
Conctructing interactive
space implies questions about the nature of installation design
in the context of shared realtime environments that are networked/distributed.
Birringers interest lies in movement experienced through images,
voice, abstract sound as well as small narrative modules or stories
or "situations" that arise from actions. Such stories may
or may not involve the body, but it is of crucial interest to investigate
networked conditions, how they are experienced and used by collective
subjects, how physical consciousness is reshaped in digital or "virtual"
environments, how people connect into these environments kinesthetically
and psychologically over time.
Performative questions for Interactive / Telepresent Space therefore
have much less to do with choreography or preformed vocabularies;
rather, they address cognitive processes:
-- how do we today navigate - orient ourselves - in the world (in
mediated space, networked space)?
- what are our means of information from which we infer where we are
('positioning system') and how we move - especially if space - as
networked condition of existence, or interdependence - is reliant
on "objects" of information gathering/locationing (cell
phones, internet, maps, interactive scenarios in everyday life, object-relations,
sensors, markers, orientation signs, sound information)?
- how do we cope with contingent, indeterminate, open, fluid "locations"
(when the meaningfulness of a location changes)?
* * *
For the architecture
of our installation design, we initially defined an approach that
places emphasis on abstract acoustic space being self-referential
as such. Our developed version of this approach is more sculptural,
referential, and sociological and involves the idea of the transobjects.
Testphase:
East by West - illusionary spaces of sound
In an initial approach,
we concentrated on a setup where visitors would be interested in making
experiences with the space-transforming sound on their own. Both rooms
are empty, the walls functioning as projection surfaces. We prepare
the most effective visuals/acoustic materials and means of transformation.
The focus of this idea
is not so much content oriented, but more situation oriented: What
would it be, if there are 2 persons, each in one (similar/identical)
room. If one person moves (bends down, waves his arms, walks around...),
the acoustic space widens/narrows. But the visitor not only controls
his/her own acoustic space, but also the visual space in the other
room. If the same is done in the other room, but crosslinked, a communication
between the two evolves. This is not a spoken word communication,
but a mediated communication: the person moves in a certain direction
to change his/her acoustic space (e.g. in order to make it congruent
with the visual perception), and at the same time the person triggers
some reaction of the person in the other room, who tries to do the
same from his/her perspective (maybe both movements contradict each
other, leading to some frustration on both sides, as the actions,
which are taken, lead to the opposite results of their intentions
due to its feedback structure). Although a seemingliy formalistic
approach this could well get interpreted in a political or social
way without having to refer to concrete images/sounds.
Prototype:
East by West - translocal situations
In the extended approach, we imagine constructing two environments
which are not identical but compose a different metaphorical ambience
or situation inviting the visitor to interact with textures and tangible
objects in the environment.
These environments create
changing imaginary space through the sound transformations, while
the telepresence connection between East and West rooms implies that
the visual images of more or less identifyable persons in these rooms
get the visitors socially and perceptionally involved as there are
some direct affective relation between them and the projected images.
The visual spaces are transformed in their physical dimensions as
well (which is central to what Finnendahl is interested in musically),
since the real-time transmissions from the other room extend the single
room, and some objects may begin to function as trans-local
props (e.g. the sand, boccia balls, oranges). Using the real-time
cross-media synthesizer allows for a range of possible manipulation
and distortion of the transmitted images, thus exploring further the
transformations of spatial realities though sound and visuals, real-time
trompe l'oeil, extensions and translocations of the room-realities.
Diagrams
IThe diagrams show the West room as a play-space with
a downward gravitational pull towards the sand-boxes and the game
of bocchia to which the visitors are invited. The West room is earth-centered
and has dim warm lighting, and at the same time might evoke explorations
of the bocchia game or sandbox playfulness in the sense in which such
a game geography involves imaginary rules.
The East room is a more abstract sculptural installation that is aerial,
showing eighty or ninety pieces of oranges hanging in the space suspended
from the ceiling and lit with spotlights from above. A broken door
also hangs under the ceiling. At first sight, the room might create
a surreal feeling, but also invite the visitor to explore the possible
functions or evocations of the suspended oranges.
As the visitor moves around, the sound in the space changes, and visual
projections will appear on both sides of the walls in each room. The
telepresence "windows" mix the presences in each room. The
projection design is under development; in later versions of this
installation we add floor projections of graphic representations of
the changing souund, activated by Eyecon tracking cameras that respond
to movement and control some of motion of the image projections and
the lighting in the space. Furthermore, the hanging objects will become
more kinetic, their behavior more unpredictable as their supended
movement (up and down) on pulleys -- activated telerobotically --
will be affected by actions of the visitors in the West room. Thus,
a kind of surreal ballet méchanique, a Bunraku-ballet of fruits
might occur in the East room.
In terms of a political or territorial imaginary, the two rooms will
be completely abstract. The underlying narratives that we wrote --
for the sandy beach/dead sea and the "Orange County" --
may never become known to the visitors. We anticipate, however, that
the real-time presence of the acoustic and visual flows, inflections,
loops, and alterations will generate certain recursive structures
and newly evolving spatial relations that may be stimulating and thought-provoking.
The real-time synthesis, performed together by the visitors, the programmers
and the controllers that activate some of the transmissions, will
generate unpredictable variations, in the constant mix and remix of
the plastic, acoustic, and projected dimensions of the continuum between
the two sites.
0.2. Participants
Johannes Birringer (USA/Germany)
Sher Doruff (The Netherlands)
Orm Finnendahl (Germany)
production assistance
Ayse Orhon (Istanbul), Sibylle Uttikal (Berlin)
Technical assistance:
Niels Bogaards (WAAG, Amsterdam),
Frieder Weiss (Nuremberg), Udo Zickwolf (Dresden), Thomas Dumke
(Dresden), Kai
and Ulf (Dresden)
0.3. Technical
plan/equipment
Audio:
2 computers with multichannel io cards and MIDI/network
1 computer for
the mapping of control data
i2 Audio Mixers
2 Amplifier
16 Loudspeakers
Video/telepresence:
Space East -
1 - G4's min 256ram
2 - video projectors (or more plus vga splitter)
vga cables
audio cables
1- usb audio interface
1 - midi interface
1 - iMac as server
1 -iMac, laptop or G4 as KS client
2- webcams
1 dv camera
Space West
1 - G4's min 256ram
2 - video projectors (or more plus vga splitter)
vga cables
audio cables
1- usb audio interface
1 - midi interface
1 -iMac, laptop or G4 as KS client
2- webcams
1 dv camera
Network
ethernet cables + hi-bandwidth connection
5 -static ip's on a subnet
the server can sit in either room or outside the rooms. same is true
for the KS clients, depending on the interaction that's chosen.
positions of cameras not yet decided
Lighting
instruments
12 lighting instruments (Leico and/or Fresnell)
12 50 or 100 ft extension cords
dimmers
Construction
materials
for the installation of two environments (lumber, glass,
steel, wires, sand, fabrics, coathangers, etc)
0.4. Budget
items
Construction materials for the installation of two
environments
(lumber, wires, sand, objects, etc)
lights (provided by host organization)
tapes, electronic components, materials
video projectors
webcams
audio equipment
(availability
or rental of equipment need to be clarified with host organization)
artistic fees
(for three principal investigating collaborators)
Additional fee for other collaborators/programmers
travel
(
TOTAL (projecetd) 18,000.00
0.5. Installation
diagrams
See diagrams on our website
0.5.0
Diagram of empty space
0.5.1
Diagram of West Room
0.5.2
Diagram of East Room
(diagrams are under development and subject to modification)
04.20.02