Intro:
My curiosity in the dynamic of sound and image is rooted in my visual art background. Specifically I've found that I often discover creative visual inspiration from meaning and emotion in music. My initial explorations in the subject basically consisted of making still images that somehow related to a song. More or less, the music would influence and inspire how I worked. In the end, the final image means something and takes on a unique meaning in context of the song (and without the song something is lost).
My excitement about sound and image isn't so different than perhaps the excitement of a cross cultural interaction between people. At some point, you may learn that you share some deep kinship with a Mongolian acquaintance only to later be shocked by how phenomenally differently they express their response. Similarly a visual artist and musician may share a common inspiration, and come to express them not just in different languages, but in different modes of language—bouncing photons and air vibrations respectively.
I find a sense of wonder in this… how can something so common and meaningful to me translate into such varied responses from others. What does it mean that I can see a whole picture at once, but I can only access sound an instant at a time? Why isn't our common vocabulary to discuss sound as sophisticated as our vocabulary to discuss cinema? What do I have to learn from people who make their dominant medium sound? What do studio artists, film composers, scientists collectively know about the mysterious interaction of sound and image together?
It's really from this mostly-uneducated sense of wonder that I developed the Sound + Image focus group to try to scratch at these things a few millimeters further beneath the surface.
At the beginning of the quarter I personally chose a few broad curiosities to pose into research questions:
- What are some of the basic tenants of music theory (in an applied sense, how does music mean)?
- What "literary" techniques in cinema require an interaction of sound and image?
- In what various ways do visual artists incorporate music/sound into their inspiration and art-making process?
- Starting with an inspiration that is neither specifically visual nor auditory, how does a sound artist imagine and produce a work? (especially in contrast to a visual artist)
A broader sense of my interests might actually be from the promo writeup I sketched out detailing possible directions for the course:
"I wanted to pass on a super cool opportunity next fall to delve into the relationship between sound and image. Do you have a background playing or composing music? Making visual art? Are you a programmer, synesthete, veejay, film maker, or physicist? Maybe you're just cool and curious? The rough plan for the course is to combine exploratory research and analysis with hands on interdisciplinary creative collaborations.
Potential course content and individual foci include:
- a foundational inquiry into the physics of acoustics and light
- research into software designed to visualize music (veejay stuff, music visualizations from iTunes, show lighting)
- visiting guests from the university and related professions (DXARTS, music composition, veejay, digital imaging)
- visual art projects to depict "sound"
- music art projects to depict "image"
- audiovisual art collaboration in response to specific narrative, emotion, or idea.
- academic research into related topics (sensation and perception, aesthetics, semiotics)
- observational analysis of sound and image in the real world
- analysis of non literal sounds in feature films (relating to narrative and image)
- analysis of image in less-literal music videos (relating to sound and meaning in music)"
Projects
Through the quarter my research deviated in the following directions:
* Background Research 1 - Audiovisual techniques of Spike Jonze (includes article summary and notes)
* Project 1 - Audio doodles (project description and directions for finding sounds)
* Project 2 - Music visualizations (research findings, framework for imagining a music visualization)
Article Title: Spike Jonze - the Music Video Director
"Jonze is One of the Most Innovative and Extremely Unique Video Directors Today."
Article Link: http://www.associatedcontent.com/article/162313/spike_jonze_the_music_video_director.html?page=7&cat=40
My Notes…
Spike Jones is an acclaimed director. all over the place (i knew him for advertisements actually) including feature film. this article focuses on music videos known for use of humor. divergent styles, witty cohesion short films linked as a playground to break norms… music videos are basically short films. In practice, music videos are a form of advertisement or branding for an artist
visuals provide a way to appreciate an artist even if they're not your styles so much. The article then breaks down three areas where it feels Jonze is especially artful, surrealism, narrative and interpretation, then highlights features of different music videos Jonze has directed that exemplify these techniques. Here they are.
Surreality
The Pharcyde - Drop
http://www.youtube.com/watch?v=co3qMdkucM0
Björk - The Triumph of a Heart
http://www.youtube.com/watch?v=nQB9d-MMIx0
[I added this one] Adidas Commercial
http://www.youtube.com/watch?v=ENleEsKhOj4&feature=related
Narrative
P. Diddy - 'It's All About the Benjamins (Rock Remix)
http://www.youtube.com/watch?v=uL8_JQzMEBU&feature=related
Daft Punk - Da Funk
http://www.youtube.com/watch?v=JZQ_vSDXXXI
The Chemical Brothers - Elektrobank
http://www.youtube.com/watch?v=PVrA2mtrHUM
Interpretation
—-> lead: "This technique is reminiscent of the way Norman McLaren uses graphic images to interpret music." Who's he and what's he about?
interpretation can be visual translation of acoustics within music, or narrative/emotional/semantic content.
FATBOY SLIM Weapon Of Choice
http://www.youtube.com/watch?v=hRZjB2zsWXY&feature=related
Björk - It's Oh So Quiet
http://www.youtube.com/watch?v=htobTBlCvUU
My thoughts spurred by the article the article:
I noticed the cross-cultural collaboration appreciation necessary for Jonze and artists to each shine. For me, I'm just feeling self-conscious watching them… what me? watching p-diddy in suzzallo espresso ;~) [good for me i think]. Throughout I felt as though the videos showed the talents of all involved shining. I had some thoughts on the dominance of the visual. It seems like the visual simultaneously elevates and competes with music… honestly the music became subconscious sometimes. I don't equate quality of sound with how dominant it is next to the visual. Visual can be so engaging that the beauty or meaning in sound is lost. In the reverse, a putting the music in context can create a powerful bridge to make an appreciation of the audio accessible.
Project 1 Description:
This project has gained focus as it has progressed. The beginning inspiration was to explore the world of digital audio manipulation. It also grew to include my interest in exploring VJ software. After broadly searching for ways to edit and process sound several pieces of software stood out.
Audacity
High C
AV Mixer
Sound Booth
Done in collaboration with Sol Hashemi, our final project seeks to stimulate an elemental awareness of sounds and images perceived as well as drawing out some of ironies of their context*. The basic stages of our project are the following:
- gathering content and learning software
- processing sound and image
- live mixing of audio and visual content
*the inspiration from this draws largely on Brian Reed's discussion of discongruence and our awareness to our perception (from class, Oct 13 08)
Most of my research consisted of discovering and learning new software. A key background component of this included learning a bit more about what exactly sound is and how sound waves interact with each other. An excellent concise summary can be found here: http://livedocs.adobe.com/en_US/Soundbooth/1.0/help.html?content=WS8D56CEB3-B638-4591-B75B-485C0B921836.html
My final product is a collection of recorded and edited sounds, which are accessible on the shared space. Link: https://catalysttools.washington.edu/sharespaces/space/danieldn/4171
(Note: files before edits have "original" in their name and after include "processed" in corresponding name)
Music Visualizations
- Researched software
- Researched components of audio and basics of how software interprets them
- Compared this with editing software I used on project one
- Found inspiring applications pushing what is visually possible based on sound analysis
Synthesis:
(What I addressed and what I'm still left curious about)
- I started to orient myself into how light and sound actually work physically. I found this exploration extremely satisfying, but very insufficent. In addition to a genuine fascination, it seems like a logical foundation to understand the medium you are working in.
- I would also like to delve into research on how they work independently and together perceptually (i.e. how do the affect you?). What do cognitive scientists, musicians, visual artists, composers, film directors collectively know? Is there a way to organize this knowledge in a meaningful way? Continuing on this, I'd like to go more in depth with Chion's book Audiovision and related works.
- Specifically with sound, I scratched the tip of the iceberg in terms of understanding what kinds of edits can be made and what the visual depictions of spectral displays and waves mean when you are editing. What I learned was interesting and I'd like to go into more depth on what makes up a sound and how its components interact. This would entail both background and experimental research… from an artistic approach to learn what is pleasing and meaningful, from a scientific to know what's physically going on.
- (In my own head at least) I opened up a philosophical discussion about how much order should be applied to mixtures of sound and image. At this point I feel that I still want both—they don't seem incompatible. I hear the arguments loud and clear about emotional manipulation and deadness by generic movie scores, in my first project I experienced a small feast of strange sounds and images in chaotic relation, and yet something in those scores still moves me and I'd like to know a bit more about why.
- As I've researched, one key missing space has to do with understanding collaboration between disciplines (returning to the example of cross cultural interactions and modes of language in my intro). Either this is an impoverished area of research or I've elegantly dodged sources without trying. It seems that, unknowns, unknowables, and bickering aside, people are generally quite active about addressing how sound, image, and their interactions affect us. On the other side of this coin however, it seems also, that there has been much neglect on studying how images are made, how sounds are made, and how individuals working in these disciplines inform and inspire each other. To me this is equally interesting and significant.
Expansion:
On the whole, I didn't stay very focused on my initial research questions, though I did stay within the broader interests I outlined in the course description. Framing these remaining interests into a year long course of study might roughly look like the following:
- Three to for months in a similar context to this class, working with artists and trying out bite sized projects. In a longer time frame, I would find it interesting to trade projects with other students to see how one person's interest/experience builds on your idea.
- Within this initial time frame, I'd also like to start compiling the theory, experience, and research of academics and artists alike who specifically deal with the interactions of making and perceiving sound and image together. While the depth might be limited, I think some new discoveries are likely to be made.
- For the next six months I'd like to build an audiovisual art piece. Components would include a "song" from sampled and edited audio, a 3D visualization which is rendered from the content of the audio, and modules for live interaction with these elements. At this point, I would lean away from completely abstracted sound and image, toward more narrative-rich content where a viewer can begin to guess (or create) a story.
