Sonification demo — Bending Energy algorithm output mapped to synthetic speech synthesizer.
Early user experience design sketch (audio)
What this system could sound like as a imagine user walks down a busy city street while listening to music.
The detection system is housed by a cell phone application which is mounted onto a pair of the virtual-reality goggles.
As objects approach the listening user, the app applies real-time signal processing to the music that describes the shape of the object.
R2-D2 as a data sonification design prototype?
“R2-D2 was the most difficult non-human character to develop a voice for, Burtt says. He was a machine that was going to talk, act, and work opposite well-known actors. But he didn’t have a face or speak English. Initial voice tests for Artoo seemed to lack “a human quality.” After some trial and error, Burtt began imitating the sounds an infant might make, and he found that it worked: R2-D2 could convey emotion without speaking words. Thus, the idea was to combine mechanical and human sounds, and Burtt combined his voice with electronic sounds via a keyboard. It helped him understand how Artoo could inflect and, ultimately, deliver a performance.”
Burtt arrived at an integration of synthesized and human speech sounds, in part through an initial process of ‘vocal sketching’ — a useful technique for rapidly prototyping sound design concepts, such as for new devices, auditory displays or systems.
More sonification design and aesthetics.
Inspired by malleable properties of the human voice.
Auditory icons like synthetic chattering voices.
Inspired partly by Domino’s Pizza scooter alert signals again.
Sounds a person cannot help but pay attention to — other than crying babies and alarms.
Fragments of conversations caught.
Brain tunes in.
Detects changing proximity and shape factor.
Brain tunes out.
Minimal cognitive taxation.
Inspired by Domino pizza delivery people in the Netherlands.
Sound design approach — less ‘stark’ (as in clinical and austere alert signals) … but more Starck (as in Philippe Starck, French designer). https://en.wikipedia.org/wiki/Philippe_Starck
Designing listening (‘choreographies of attention’) rather than designing sound.
Diagram of the hardware components of the user’s system:
• Pair of headphones wired up with a binaural microphone at each ear.
• Enables the user to monitor both external auditory environment while listening to prerecorded audio (music, podcasts, etc.)
• A cell phone app mixes the different signals automatically according to the detected presence and shape factor of objects that enter the user’s detection system.
Mind mapping sonification variables for experimentation.
For next meeting — we will assemble a various of small moving objects to document on video for purposes of experimentation with sonification through computer vision.
▪ tracking and estimating movement speed and trajectories of the objects,
▪ How to identify important (visual, mass, texture, weight) properties of a static object
▪ Identify #1 but while object is moving.
Track moving object as it moves towards the path of object detection system of visually impaired user. In which the listening sister boundaries once the moving object enters in to that within that boundary the listeners system will detect it and translate both its shape and its proximity to sound.
I’ve been working with Bluemouth Inc these last few weeks on the first breath of a new immersive performance project – so far a kind of hallucinatory take on ‘summer camp’ – through a short residency at the Gibraltar Point Center for the Arts, Toronto Island. I think the last time I worked on an art project here was with electroacoustic composer Darren Copeland (New Adventures in Sound Art). We were making a radio documentary about the history of Toronto’s soundscape, on the island, in the dead of winter, 1999. The island is an insanely beautiful place to come to work to every day. Everything about it – the constant surf crashing against the shore, the expansive blue sky, the wind that rushes through everything – it all insists that you leave what’s past behind you and move forward. Although a lot of the things we experimented with during this residency reminded me (perhaps too much) of many past (and in some cases very early) projects, the island itself insists that I am only as good as my last work and so would I please mind moving forward and not repeat myself?