On Sound Design for the Multi-Sensory Museum

Keyword:

‘perception’ + ‘recipient’ = ‘percipient’

Why sound?

For visually impaired percipients, sound is one of several alternative perceptual modes through which an artifact may be experienced.

For both the visually impaired and the visually enabled, the addition of sound can serve to lengthen the duration of perceptual engagement with an artifact. Just as the process of composing a piece of music can be thought of a process of structuring someone’s experience of sound through time, the process of designing an auditory component for an artifact may be thought of as a process of structuring someone’s experience of an artifact through time.

Possible Approaches to Explore

1. Sonic Accompaniments (‘Framings’)

Accompaniments can convey aspects of an artifact’s history, cultural contexts and the circumstances surrounding its creation and presentation through sound.

They could include recordings of the artists themselves discussing their work or other voices reading writings by or about the artists, discussing aspects of the artifact’s historical, cultural, social, economic or political significance.

As such, they may be thought of as the auditory equivalent of a display case or picture frame which surrounds the artifact with (what might be called) contextual narratives and/or third-party interpretations. A familiar example of this can be found in many art museums in the form of hand-held audio guides that patrons can listen to while moving through the gallery.

2. Sonic Augmentations

I tend to think of this approach as less concerned with framing the artifact with narratives and more concerned with evoking aspects of the artifact’s materials, production methods and environmental surroundings through sound.

It focuses on the creation of soundscapes in which pre-recorded or simulated sounds of the artifact’s composite materials (i.e., canvas, wood, paper, metal, stone, etc.), the tools and processes used by the artist in making the artifact (pencils, brushes, knives, hammers and chisels, fire, water, steam, etc.), and the surrounding sonic environment within which the artifact was created all serve as the principle sonic materials.

Augmentations may be presented as pre-recorded soundtracks or—if you have the time and materials—as interactive, exploratory experiences. Imagine, for example, that a percipient holds in their hands a replica of a particular artifact. As they examine and manipulate the artifact, a sequence of sounds derived from its physical and acoustic properties become audible to the percipient.

3. Sonic Translations

Of these three approaches, I think this one speaks perhaps most directly to the ambitions of inclusive design.

The purpose of a sonic translation is to convey the details of an artifact’s 2D (graphical, textual, chromatic) or 3D (physical, spatial) properties entirely through sound, such that a percipient can begin to form a mental model of the artifact through the act of listening alone.

This approach is often derived from methods of data sonification and involves a process of mapping and translating the visual or spatial properties of an artifact to a corresponding set of sonic properties.

These sonic properties are often referred to as variables — properties of sound which change in accordance with changes in the artifact’s spatial or graphical properties.

Prototyping Techniques

These are all rapid prototyping techniques intended to get you working with sound, interactions and experiences quickly.

For more on these techniques and the broader field of sonic interaction design, I highly recommend starting with the materials cited here: sonic interaction design (Wikipedia) and the book Sonic Interaction Design (2013).

1. Vocal Sketching

Within the context of a design project, vocal sketching is a low-cost means of rapidly prototyping initial sound ideas.

This is because the human voice (including yours) is perhaps the most versatile instrument we have for mimicking the sounds of our environment. In fact acoustic mimicry (a component of biomimicry) has been a key part of many species (including humans) survival and persistence through the evolution of this planet.

As an experiment, try to imitate the following sounds:

a dog’s bark
a cat’s meow
a car crash
Laser gun firing
Rainfall
an alarm clock
a mosquito
a creaky door
a wind storm
a police siren

Next, explore ways of morphing or transforming one of these sounds into different sound from the list.

Then try recording your vocal sketches into an audio recording and editing application such as Adobe Audition, Audacity, GarageBand and then experiment with editing, layering, and processing the sounds in order to produce entirely new sounds.

But remember, these are supposed to be sketches—not the finished product.
In the early stages of a design process, the sounds you make do not have to be completely convincing—they just have to get the point across.

2. ‘Foley’ Sound Effects Techniques (Exploring Audio-Tactile Forms of Engagement)

Another creative approach to imitating sounds from the world around us is through the use of ‘foley’ sound effects techniques developed by the motion picture industry.

This approach focuses on manipulating physical materials and objects in order to imbue movies with heightened degrees of sonic realism. Typically, these interactions are performed and recorded in time to the film.

Notice how much of a performance the foley artists themselves seem to be creating while tracking a film sequence.

So why not experiment with audio-tactile interactions wherein the percipients themselves engage in creating sounds through ‘foley’ materials and techniques (for example, manipulating a pile of fallen tree leaves to produce sounds as well as olfactory and tactile sensations)?

Here are a couple links to webpages containing ‘foley recipes’:

27 Secret Tricks and Ideas for Foley Sound Effects on Your Film

Foley Sound Effects from Everyday Household Objects – Storyblocks Blog

3. Video Prototyping With Sonic Overlays

This is a technique for rapidly prototyping sonic interactions without having to configure sensors, interface hardware and computer code.

This approach lets you focus more on the user/percipient’s interactions and the kinds of sounds those interactions produce.

In this example—a project titled Sound and Pepper by students at the Holon Institute of Technology—the design team made a video of the intended user interactions (manipulating various spice containers while cooking) and then developed the sounds for each container. Video editing software was then used to match each of the sounds to the physical gestures in the video:

A typical workflow might be to capture demonstrations of possible interactions with the artifact on video using a smartphone, laptop or web camera. Then import the video to an editing application (Adobe Premiere, iMovie Wondershare Filmora, Final Cut Pro X, etc.) and then record (overdub) sound effects in time with the video interactions using simple foley techniques.

Below is another example of prototyping with sonic overlays. It was produced through a collaborative design research project between myself, the Biomedical Simulation Lab (Dr. David Steinmen, Director; University of Toronto), and the Perceptual Artifacts Lab (Dr. Peter Coppin, Director; OCAD University).

The project’s aim was to develop new audio-visual interfaces for clinicians attempting to diagnose blood clots (aneurysms) in their patients by translating blood flow data into sound.

Initially, the biomedical simulation team would rely on vocal imitations of fluid flow sounds heard through ultrasound to convey a sense of what sounds would be most meaningful to a clinician. Drawing on these imitations, I created a sonic overlay to accompany this computer animation of blood flowing through an aneurysm.

This approach enabled the team to temporarily bypass the more time consuming tasks of extracting, processing and mapping fluid flow data to computer-generated sounds and focus more on developing a convincing palette of sounds that could be potentially meaningful to a clinician.

4. Physical Prototyping

Physical prototyping focuses on realizing sounds and interactions with higher degrees of fidelity using electronic sensors, interface hardware (Arduinos, cameras and other sensors), and interactive software applications (Processing, Pd (Pure Data), Unity, MaxMSP, SuperCollider, etc.).

Ideally, these techniques are deployed after one or more initial design stages involving rapid prototyping techniques (vocal sketching, ‘foley,’ videos with sonic overlays, etc.) have occurred.

Here, for example, is a subsequent iteration of the Sound and Pepper project, realized through physical prototyping techniques:

And here is a subsequent iteration of the blood flow sonification project in which a video game controller manipulates playback of my sonic overlay using MaxMSP software. The purpose of this prototype was to explore possible user interface designs wherein a clinician can isolate and explore different regions of an aneurysm through sound:


Case Study: Towards the Sonic Translation of a Two-Dimensional Infographic Artifact

This project was developed while working as a research assistant in the Perceptual Artifacts Lab (OCADU) and the Biomedical Simulation Lab (University of Toronto), 2017.

The central aim of this project was to develop strategies for translating the graphical or diagrammatic properties properties of an infographic map into an equivalent auditory experience, such that a visually impaired percipient would be able to construct a reasonably accurate mental model of both the infographic map and its informational content through sound alone and without the use of text-to-speech software.

For this project, we selected Charles Minard’s pioneering 1869 infographic Figurative Map of the Successive Losses in Men of the French Army in the Russian campaign 1812–1813:

Charles Minard’s pioneering 1869 infographic Figurative Map of the Successive Losses in Men of the French Army in the Russian campaign 1812–1813

An early precursor to contemporary data visualizations, this infographic map depicts Napoleon’s ill-fated attempt at invading and conquering Russia in 1812.

In its time, this map was remarkable for its ability to compress the following six dimensions of data into a two-dimensional representation:

  1. troop quantity
  2. geographical distance travelled between the Poland-Russia border and Moscow
  3. the changing meteorological temperature from June 1812 to January 1813
  4. the changing troop locations (in terms of latitude and longitude)
  5. changing travel directions of the army
  6. location­-specific dates and landmarks ­

Here’s a breakdown of the infographic map’s properties:

This map also exemplifies how a two-dimensional visual representation can effectively distill the informational content contained within a 574 word, text-only description.

Compare your experience of viewing the infographic above with reading the text-only description below. Pay attention to how they differ in terms of time and cognitive processing.

“In the Russian Campaign, the French Army started at the city of Kowno near the Neiman River, with 422,000 men advancing in the 2 o’clock direction. Halfway towards the city of Wilna, 22,000 men split off towards the north. The remaining 400,000 men continued advancing towards Wilna in the 4 o’clock direction. At Wilna 66,000 men split off towards the northeast. The remaining men continued advancing towards the city of Witebsk in the 2 o’clock direction. At Witebsk 175,000 men remained. They continued advancing towards the city of Smolensk in the 4 o’clock direction. At Smolensk 145,000 men remained. They continued advancing towards the city of Dorobobouge in the 2 o’clock direction. Then they continued advancing towards the city of Chjat towards the northeast. At Chjat 127,000 men remained. They continued advancing towards the city of Mojaisk in the 4 o’clock direction. At Mojaisk they crossed the Moskva [Moskowa in original] River, and 100,000 men remained. They continued advancing towards the city of Moscow [Moscou in original] towards the northeast. At Moscow 100,000 men remained. They started retreating southwards towards the city of Malo-jarosewli, via the city of Tarantino. On October 18 they reached Malo-jarosewli and the temperature was zero degrees Réaumur. The remaining 96,000 men took a short turn towards the southwest. On October 24, about halfway towards the city of Mojaisk, it rained. They continued retreating towards the city of Mojaisk in the 10 o’clock position, but the temperature started dropping. After reaching Mojaisk, the remaining 87,000 men continued retreating towards the city of Wizma to the southwest, After reaching Wizma, the remaining 55,000 men continued retreating towards the city of Dorogobouge to the southwest, On November 9 they reached the city of Dorogobouge. It was -9 Réaumur and the temperature started to drop more sharply. The remaining 37,000 men continued retreating towards the city of Smolensk in the 8 o’clock direction. On November 14 they reached the city of Smolensk. It was -21 Réaumur but the temperature started to rise. The remaining 24,000 men continued retreating towards the city of Orscha in the 8 o’clock direction. They crossed a river near Orscha, which was some distance north of Mohilow. The remaining 20,000 men continued retreating towards the city of Botr in the 8 o’clock direction. When they reached Botr it was -11 Réaumur, and the temperature started dropping again, but 30,000 men from the second split rejoined them. These 50,000 men continued retreating towards the city of Studienska in the 8 o’clock direction. On November 28 they reached Studienska near the Bérézina River, and it was -20 Réaumur. The remaining 28,000 men continued retreating to the northwest. On December 1 they reached somewhere north of Minsk, and it was -24 Réaumur. They continued retreating towards the city of Molodezno towards the southwest. On December 6 they reached Molodezno. It was -30 Réaumur, but the temperature started to slowly rise. The remaining 12,000 men continued retreating towards the city of Smorgoni in the 10 o’clock direction. At Smorgoni the remaining 14,000 men continued retreating towards the city of Wilna to the west. On December 7 8,000 men reached Wilna, and it was -26 Réaumur. They continued retreating towards the city of Kowno to the west. About one third along the way 4000 men remained. About two thirds along the way 6000 men from the first split rejoined them. These remaining 10,000 men continued retreating towards the city of Kowno to the west.”

Introduction to the Concept of Visual Variables in Data Visualization and Sonic Variables in Data Sonification

To illustrate what is meant by ‘sonic variables,’ here is a set of visual variables (position, size, shape, value, colour, orientation, texture) developed by early data visualization researcher Jacques Bertin in the 1970’s:

Bertin’s visual variables.

Today, these are widely accepted as the basic building blocks or vocabulary for creating visual representations of data for purposes of exploration and interpretation.

By comparison, here is a set of sonic variables (location, loudness, pitch, register, timbre, duration, rate of change, order, attack/decay) developed in 1994 by cartographer JB Krygier:

Table of Krygier’s sonic variables.

Today, these are widely accepted as the basic building blocks or vocabulary for creating data sonifications or auditory displays of information.

Design Prototypes

Sonifying Napoleons March Using Auditory Icons (“Audio Play”)

My earliest iteration, this is an audio-only sonification of the data collected and depicted by Minard in the original infographic.

In this iteration, pre-recorded foley effects (footsteps on a variety of surfaces) are combined with stereo panning and frequency filtering to convey changing troop quantities, troop direction, and geographic landmarks.

A single pair of footsteps represents 1000 troops. The footsteps were layered many times and mixed together in order to approximate the total size of the army (422,000 troops) at the beginning of the invasion.

Note: One as-yet-to-be-realized iteration of this project would be to involve percipients as participating live foley effects artists who follow a video-projected score to produce a participatory sound performance (or, I suppose, an info-sonic map).

Data-to-Sonic Variables Mappings:

  1. Kowno (Poland-Russia border) => far left stereo position
  2. Moscow (Russia) => far right stereo position
  3. Troop count => 1) quantity of footsteps; 2) field drum tempo (slower tempo = fewer troops)
  4. Terrain (seasonal) => footstep surface texture (i.e. winter = marching through snow sound effects)
  5. Temperature (during retreat phase only) => sine wave

Note: This sonification adheres to a strict temporal structure (1 second of audio = 1 day of the army’s campaign) whereas Minard’s original infographic does not.

Sonic Translation of Graphical Properties

This prototype explored how variations in the range of audible noise frequencies might be able to convey information about the changing line thicknesses both vertically and horizontally across the image through sound alone.

As the red slider is scrolled back and forth across the tan and black lines of Minard’s infographic map, the changing pixel colour data along the vertical (Y) axis determines which filter bands will allow white noise to pass through at any given point on the the surface of the map. At the same time, the stereo position of the resulting sound is mapped to the horizontal (X) axis.

Prototype Demonstrating Interactive Data Sonification of Charles Minard’s 1869 Infographic “Figurative Map of the Successive Losses in Men of the French Army in the Russian Campaign 1812–1813” Through Graphical Navigation.

Sonification Demo: Napoleon’s March Sonification Through Graphical Navigation from Richard C. Windeyer on Vimeo.

In this iteration, I retained the use of frequency-specific band-pass filters and stereo positioning as a way to convey spatial position across the surface of the image, but combined this with the marching footsteps sound effects used in the earlier prototypes.

A pure tone was also added to depict the falling temperatures endured by the troops during the retreat phase of the campaign.

Intended for visually-impaired percipients, the spatial properties of Minard’s two-dimensional infographic are conveyed through computer mouse cursor position onscreen.

Data-to-Sonic Variable Mappings:

  1. Kowno (Poland-Russia border) => far left stereo position
  2. Moscow (Russia) => far right stereo position
  3. Troop count => marching tempo (slower tempo = fewer troops)
  4. Terrain (seasonal) => footstep surface texture (i.e. winter = marching through snow sound effects)
  5. Latitude/Longitude =>centre frequency/pitch of white noise band
  6. Temperature (during retreat phase only) => sine wave

Note: This sonification adheres to a strict temporal structure (1 second of audio = 1 day of the army’s campaign) whereas Minard’s original infographic does not.

This particular iteration was valuable for the many questions it provoked about the peripheral aspects of perception — and the tendencies for percipients to lose their sense of spatial position while exploring the prototype.

This iteration also led to a greater focus on player experiences with audio-only video games—particularly those games that used 360-degree surround-sound encoding as a means of immersing players in the gaming environment. We began to refer to this use of immersive, surround-sound encoding as ‘the World of Warcraft metaphor.’

Sonification Prototype 2.0: A Binaural Translation of Charles Minard’s Infographic Map

In this iteration, the sound of marching footsteps are indexed and mapped within a virtual surround-sound auditory space designed to exploit a fundamental affordance of human hearing: the ability to decipher information within an environment through the localization of discrete sound events within a 360 degree radius around the percipient.

The intention of this prototype was to develop strategies for translating the graphical properties of a map into a virtual auditory space for use by visually-impaired individuals.

Precise quantities of troops and the names of nearby cities are represented through text-to-speech synthesis. These are activated by tapping designated keys on a computer keyboard (‘q’ for troop quantities; ‘w’ for locations)

During the production of this iteration, much of our conceptual work had begun to focus on acquiring a better understanding of how different kinds of sonic variables serve the task of representation terms of semiotics (the study of signs and symbols and their use or interpretation) and auditory cognition. Much of this thinking involved consolidating scholarly research in the field of auditory display design (Kramer; Walker; Keller & Stevens; Gaver).

The sounds of footsteps, marching drums, church bells and military battles draw upon ecological resemblance (the sound resembles the thing it represents either directly or indirectly).

By contrast, the descending tone representing temperature change is considered symbolic, meaning that the sound represents something through the establishment of a convention which is to be made clear to the percipient prior to their engagement with the representation.

Yet as design researcher Bill Gaver observed, direct/indirect ecological and symbolic mappings are not distinct or separate categories but are really just points positioned across a representational continuum, with the majority of mappings used to depict data through sound tending to fall between these points (Gaver, 1986).

For example, direct ecological mappings are usually easier to learn because they draw upon a percipient’s pre-existing experiences and interactions with sound.

By contrast, indirect ecological or metaphorical mappings are somewhat harder to learn, and symbolic mappings are the most difficult of all. (Gaver, 1986)

Sonification Prototype: a binaural translation of Charles Minard’s infographic “Figurative Map of the Successive Losses in Men of the French Army in the Russian campaign 1812–1813” (1869) (version 2.1)

Sonification Prototype: a binaural translation of Charles Minard’s infographic depiction of Napoleon’s Russian campaign from Richard C. Windeyer on Vimeo.

This final iteration attempted to exploit how percipients can develop an ability to estimate the quantity of some thing inside a container just by shaking the container and interpreting the balance between the hollow, resonant sound of the container and the sounds of the objects rattling around inside.

For this iteration, I envisioned a percipient shaking a wooden or metal box containing a quantity of tin toy soldiers rattling around inside.

Could a percipient obtain an impression of the rate at which the troops were diminishing?

Simultaneously, how would the physical act of shaking a box filled with toy soldiers influence or colour a percipient’s experience?

Would the knowledge that the box contains toy soldiers and that the subject of the work is Napoleon’s failed attempt at invading Russia soldiers provoke associations and interpretations between the notion of toy soldiers and how Napoleon may be regarded his troops when he sent hundreds of thousands of troops into an inhospitable terrain without appropriate supplies, uniforms or anticipation of the coming Russian winter?


Playback and Presentation

Headphones (for single-person listening experiences)

Loudspeakers

Small, portable, wireless (Bluetooth)
Installations using just the bare speaker drivers (cones):

Canadian-German sound installation artist Robin Minard:

Playback sources:

  • mobile devices, iPods, laptops,
    • Recordable sound chips:

Examples of Sound Installation Art


Suggested Reading

  • Bregman, A. S. (1990) Auditory scene analysis. MIT Press: Cambridge, MA
  • Farnell, Andy. Designing Sound. MIT Press, 2010.
  • Franinović, Karmen et al. Sonic Interaction Design. MIT Press, 2013.
  • Gaver, William & W, William. (2020). Everyday listening and auditory icons. Thesis (Ph. D.) University of California, San Diego, Department of Cognitive Science and Psychology, 1988.
  • Gaver, William W. “What in the World Do We Hear?: An Ecological Approach to Auditory Event Perception.” Ecological psychology, vol. 5, no. 1, 1993, pp. 1-29.
  • Hermann, Thomas et al. The Sonification Handbook. Logos Verlag Berlin, 2011. https://sonification.de/handbook/.
  • Keller, P., & Stevens, C. (2004). Meaning from environmental sounds: Types of signal-referent relations and their effect on recognizing auditory icons. Journal of experimental psychology: Applied, 10(1), 3.
  • Schafer, R Murray. The Soundscape: Our Sonic Environment and the Tuning of the World. Simon and Schuster, 1993.
  • Truax, Barry. Acoustic Communication. Greenwood Publishing Group, 2001.
  • Walker, B. N., & Kramer, G. (2005). Mappings and metaphors in auditory displays: An experimental assessment. ACM Transactions on Applied Perception (TAP), 2(4), 407-412.

Fingers revived

Performance: Finger Reunited from richard windeyer on Vimeo.

Cameron McKittrick (theremin, laptop) & Richard Windeyer (drums, laptop) at ‘Wired’ (Laurier Music Festival, Wilfrid Laurier University, January 31, 2016)

Last January (2016), Cameron McKittrick and I revived our digital performance project ‘Finger’ for an alumni reunion concert at the Wilfrid Laurier University Faculty of Music. Once upon a time, Cam and I were both composition students there. More recently, we both worked there as sessional instructors in composition and music technology — which is essentially how the Finger project came to be.

In this performance, Cam’s instrument consists of theremin-controlled piano samples which are activated and organized through MaxMSP software. My drum kit uses acoustic MIDI trigger data to manipulate a granular synthesis engine containing a vocal improvisation recorded by longtime friend, collaborator, and spoken word artist, Angela Rawlings.

Although our improvisation was brief, it seemed to succeed as a performance — or so it seemed based on post-concert conversations with audience members. This was (I suspect), partially due the fact that we intentionally chose NOT to focus our attention on the screens of our laptops, something which has become a persistent condition of so many laptop-based performances. Instead, we focused on anticipating, interpreting and reacting (sonically) to each other through our respective physical (especially facial) gestures.

It seems to me that one primary consequence of any performance that employs computers is that it renders far too much of the actual ‘work’ of performance invisible to the viewer. Our solution here, simply put, was to transfer the perception of that work to the interplay of two human bodies struggling to formulate — and then exchange — informational cues in realtime.

Much more to be explored here (I think, I hope).

Links:

The Hunger at Harvest

The Hunger – a multimedia installation by interdisciplinary artist and designer Margaret Krawecka was recently adapted for presentation in the heart of a forest at the (ever-inspiring!) Harvest Festival (Autumn Equinox Arts & Music Festival) held every September at Midlothian Farm, Burk’s Falls, Ontario.  While technical limitations forced a scaling-down of my sound design for it, all went very well. You can find more photos and full production credits here.

 

Sound design for ‘The Hunger’

Immediately following the DANCE MARATHON performances I dove straight into completing the sound design work for a performance installation by Margaret Krawecka / Uncanny House Collective – The Hunger.

Image

The Sea Museum

Entrance to the tunnel at Atlantic Ave. and Court St., BrooklynA little while ago, Bluemouth associate artist Daniel Pettrow was invited by the French Institute Alliance Française (FIAF) to stage the first English reading of French novelist Marie Darrieussecq’s play Le Musée de la mer [THE SEA’S MUSEUM] during its annual Crossing the Line Festival of inter-disciplinary contemporary arts, presented this month in New York City. Daniel subsequently recruited Bluemouth Inc to help realize this staging in – of all places – the abandoned subway tunnel at the intersection of Atlantic and Court  Ave in downtown Brooklyn. Thanks to Dan and FIAF, I was able to fly down to NYC last friday morning and help prepare the sound design for the play’s readings on saturday and sunday afternoon.

Down the tunnel

Although New York designer Omar Zubair and I had already generated a nice collection of sonic gestures and textures over the past few months (including a generous donation of Icelandic soundscape recordings by Toronto poet Angela Rawlings) , the incredibly focused acoustics of this half-mile long tunnel steered us in a more austere direction. As over 70 people descended the ladder one at a time from street level down into the tunnel, I mashed up vintage Hawaiian music with the soundtracks of Jean Painlevé’s early aquatic documentaries. A large video projection of Jean Painlevé’s films spanned the tunnel’s curved brick ceiling and enveloped the waiting audience.

Pettrow reaches the endWith the actors leading the audience by flashlight, the play unfolds  down the entire length of the tunnel. Long, deep and slow moving wedges of sound slowly rise and follow the audience as they move towards a dead end. In setting and adjusting sound levels I would leave the mixer station and walk halfway down the tunnel, only to discover that volume level of the music was not changing at all. Higher frequency content would naturally disappear with distance, leaving only a juggernaut of low end to continue soaring down through the tunnel. Speaking and Ciara Adam’s singing at one end of the tunnel were clearly audible at the other end. We hardly needed amplification at all.

The Centre for Sleep and Dream Studies

The Scream Literary Festival 2010Last Wednesday night I returned to The Centre For Sleep and Dream Studies by way of Levack Block’s front room bar in Toronto. This version of “The Centre…” took the form of a late-night, 4-hour interactive audio-lounge surreality event curated by Angela Rawlings for The Scream Literary Festival.

I first entered The Centre almost 5 years ago as part of a creative team assembled by Angela to help investigate how her book, Wide slumber for lepidopterists, might translate from page to stage.

Here is Angela’s own “Report on The Centre for Sleep & Dream Studies, + Somniloquixotic Questionnaire” from last week’s re-visiting.

… and here is an excerpt from the night’s audio highlight – an improvised performance by Angela Rawlings and Ciara Adams (vocally improvised sounds of breath, fricatives, song, and orgasm) and Richard Windeyer (live electroacoustic processing)

While the event ended well, I have to say this was a crazy (but insightful) gig.

Angela and I had met the week before to plan out what “The Centre” would sound like over the course of its 4 hour performance. We used the different stages of NRem sleep to create a loose temporal structure, collected a variety of audio sources (live interviews with audience members on the subject of their dream experiences, the nocturnal utterings of Dion McGregor, excerpts from Gordon Jenkins classic recording “Seven Dreams (A Musical Fantasy)”, pop songs of sleeping and dreaming and so on.

Now, we were expecting to perform this in the back room of the venue to a fairly captive listening audience. However, upon arriving at the bar, we found our gig re-located to the front room bar, which was now waist-thick in a very frantic and hungry post-reading chatter party vibe. Suddenly the room for the kind of sonic detail and nuance we had emphasized in our planning was gone. Obliterated by a crowd starved for party.

The sound system was basic and the overall noise level in the room intense. In retrospect, what would have really helped me take control of the room, was a free-standing DJ set in my back-pocket, with all tracks fully beat-mapped, warped and indexed – just to keep the party bouncing while our well-made plans started to self-combust. Unfortunately I had come prepared with something very different. So after dashing back and forth between my prepped tracks and stuff on my iPhone, I let the groove settle on a very stripped down texture of beats (built gradually by hand/mouse), oscillating low-end bass patterns, randomly looped and  vocoded fragments from the dreamer interviews happening at the back of the room. In the end, this approach seemed to work, largely by establishing a rhythmic framework or counterpoint, through which, all the party chatter (and the surreal dreamings of Dion McGregor and audience members) could be heard. The trick was in knowing to leave (a lot) of space for every other sound in the room, working with it as a foregrounded texture, rather than pick a fight with it. I know many other artists and DJ’s who would have handled the situation with much more assertion (even sonic aggression). But this seemed to work – at least for this (nicely niche) crowd – evoking mostly positive responses (thanks everyone!)

I would love to try this approach again sometime.

(though I still resolve never to show up to the gig without something in my back pocket!)

rEDwIREaRCHaNGEL

rEDwIREaRCHaNGEL (2010-2011) is a conducted freepolyfunk big band project conceived and directed by Toronto musician Nilan Perera. It’s musical subject is the early electro jazz recordings of Miles Davis – ‘Bitches Brew’, ‘On the Corner’ etc.

Simeon Abbott- Keyboards/tapes
Bryant Didier – Electric bass
Jeremy Strachan – Sax
Dan Gooch – Trumpet
Rebecca Hennessy – Trumpet
Jesse Levine – Keyboards
Germaine Liu – Drums/Percussion

Nian Perera – Guitar/Conductor
Jeremy Strachan – Woodwinds
Richard Windeyer – live sampling/dub effects
Mark Zurawinski – Drums/Percussion
Ronley Teper – voice

I joined this band a little while ago, not as a drummer, but as a live dub and sample artist. My function is to loop, sample and process (filters, echoes, modulators) the band LIVE (!), in ways that mirror the great producer Teo Macero’s studio manipulation techniques, which he used to (literally) construct these classic (and pioneering) albums. The basic idea is that my activities should complement Nilan’s activity onstage as a ‘conductor’ – steering the individual musician’s output in ways that emulate the ‘tape edit’ aesthetic of the original recordings (It’s similar in spirit, I suppose, to the ‘jump cut’ techniques John Zorn employed in pieces like ‘Spillane’). I was attracted to this project initially because, at the time, Miles was listening not only to Jimi Hendrix, but also the electroacoustics of avant-garde composer Karlheinz Stockhausen. At the same time, the pioneering dub producers (King Tubby, Lee Perry…) were also digging into the very same territory. In the context of my own work (ambient/dub/psychedelic/live electronic performance), all of this represents (for me) a critical and vital intersection of influences and sonic territories.

Hear rEDwIREaRCHaNGEL on Myspace

Finger performance (an introduction)

Since 2002, I have been collaborating with Cameron McKittrick and Leslie Wyber under the name of FINGER, a live electroacoustic performance trio creating new work and re-interpreting old works by mixing fresh compositional approaches with new performance technologies. We have become increasingly interested in the impact of electronic mediation on live performance – especially where it concerns the perception of physical gesture, interaction and issues of scale. In our recent work we have examined the role of mediatization in performance forms.

A common obstacle in creating convincing electroacoustic performance concerns the use of laptop computers as instruments — their computational (and compositional) power hugely outweighing their corresponding visual appeal as instruments in live performance. The traditional instrumentalist’s large and culturally familiar gestures are in stark contrast to the visual component of a typical laptop computer performance:  In a laptop performance, the audience regularly reports frustration resulting from their inability to ground the sound they are hearing in the actions they are seeing. Past FINGER performances have often evoked experiences similar to that of sitting in a live radio audience watching the small physical actions of the sound effects artist become transformed into much larger, sonic images. For example, how the tapping of coconut shells comes to represent horse hooves in the mind of the listener. More recent FINGER performances – focusing specifically on the intense amplification of small, manually performed aural and visual gestures – have made us aware of the audience’s need for a larger performative context.  At the same time, we are increasingly aware of the risks involved in amplifying an audience’s incredulity through fantastic gestural interface without apparent governing artistic intention.

( ( (cocktail party effect) ) )

Audible hors d’oeuvres for large social functions

(   (  ( cocktail party effect )  )   ) is a performance installation series intended for large social functions, a series of fun, surprising and entertaining interventions appear from, and disappear quickly into the crowd.

Antique black and white photo of waiters in formal attire

Audio Waiters premiered as a ‘guerilla performance’ in 1999 during the intermission of a NUMUS concert in Waterloo. The central themes explored in this work included the use of music in restaurants, fine dining as a theatrical experience, the mass consumption of music (disguised as hors d’oeuvres) by pre-occupied listeners engaged in the larger social context of music concerts, and a consideration of the ‘service staff’ role which working musicians have typically found themselves playing in Western societies throughout history.

Inter Arts Matrix has presented (   (  ( cocktail party effect )  )   ) in many unusual venues, including the National Ballet of Canada Galas (2008-2009) and the Stratford Shakespeare Festival Fundraisers (2008-2009).

Performed by:
Pam Patel, soprano
Margaret Báardos, mezzo soprano
Michael Donovan, baritone
Jamie Hofman, baritone
Donovan Locke, tuba, multi-instrumentalist