track 1

track 2

PROCESS

• pick tracks
  • i selected two minimal sounding tracks (with recommendations from Mimi). why minimal? more clear to how they make me feel i.e. easier to tune in.
    • track 1 – Three Voices by Morton Feldman
    • track 2 – Prayer by Huun Huur Tuu
• tune in to how i feel
  • this was a lot more difficult than expected. even though i am a non-musician, when trying to ‘listen’ to something, i often internalize it logically i.e. in terms of notes, pitch, rhythm etc. perhaps because there are the visuals associated with those i.e. sheet music, spectrograms, itunes visualizers.
  • picking minimal tracks helped a lot with this – more single-minded
• sketch out on paper
• look at data from track
  • fft analysis – analyses the amplitude at different pitches, pick number of bins from 16-1024, pick smoothness / noisiness
    • try to visualize this in different ways to see if there are patterns corresponding to my sketch on paper / how i feel
       
      
    • this is the fabric of all things, spotting the right patterns can lead to very satisfying results
  • get energy – take ranges of bins and averages out the values
    • this is a lazy way … although sometimes this works well, it lacks the nuances which sometimes give insights to the tracks … because it ‘averages’ out the data.
  • amplitude – overall volume
    • since it is an ‘overall’ data, this works well when used to set the ‘tone’ of the drawing, equivalent to a canvas’s quality or primer in a painting.
• start visualizing (coding)
  • this process is full of wonderful surprises // numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships,
  • time plays an important role in making you feel something (much like in animation) … frame rates, increments, decrements // factor this into numbers and relationships
• fine-tune visuals
  • fine-tuning takes the most time – changing the transparency by 1% can change the whole drawing. this is a lot of going back and forth between how i feel from listening to the music and what i see
• know when to end
  • when your ears and eyes feel like they match. i find this process a lot similar to a painting, going too far makes the sketch look ‘overdone’
  • caution: do not get carried away by data – must go back to tuning in process // it is very easy to be lured by the sophisticated visuals (this happens often when headphones are off and i’m just making the sketch ‘look nice’)

NOTE TO SELF

• the best advice i received was to ‘just put in the hours’ // it is like trying to learn illustrator’s black and white mouse tools (so weird)

CONCLUSION

• this exploration started because i wanted to hear what musicians were hearing, with a hypothesis that understanding the technical aspects of it and realizing its patterns would help me appreciate and ‘hear’ music more. however, through the exercise i realized it wasn’t about what went in to create the music, rather it’s about listening and realizing what and how exactly a particular sound makes me feel.
• when listening to music, the feelings i get do not translate to spectrograms, rather some abstract, undefined … stuff
• these visuals work as a bridge to help me define and express my feelings of a musical piece
• i hope these visuals stimulate people to ‘listen’ and that they bring people closer towards their feelings (that they might or might not know are there) when they listen to a piece of music.
• conclusion code: *???* i like arrays because they are like a flexible and well organized box of materials 

PROJECT FILES & WEB EDITOR LINKS

project files – tracks, codes, visual thumbnails, other notes // https://drive.google.com/drive/folders/1XkQERxy_yp-saI_JC0BDk5wt1nVYoFUc

notes from playtesting // https://hellonun.blog/2018/11/23/playtest-sound-frankenstein/

track 1

• track 1
  • play track 1 – how do you feel?
  • play visuals for track 1 – can you see what you feel from what you hear?
• track 2
  • play track 1 – how do you feel?
  • play visuals for track 1 – can you see what you feel from what you hear?

POST-PRESENTATION QUESTIONS FOR CLASS

• how do you feel?
• what are the similarities and differences between what you see and what you hear?
• what if you were to visualise the tracks? what would it be like?

FUN LINKS

interesting thing about frequency // wine glass breaking https://www.youtube.com/watch?v=z6oqPB07X3o

everyday sounds we look pass (or hear pass) // sonic branding https://www.youtube.com/watch?v=S_gBMJe9A6Q

it’s about communicating emotions // comment on jacob’s video – https://www.youtube.com/watch?v=eRkgK4jfi6M

what actually happens when you create a sound – the visual is more like this than like notes or other metrics // tom thum – https://www.youtube.com/watch?v=LqdFL0u2HLY

reason to visualise based on feels // feldman site http://www.cnvill.net/mfhome.htm 

reason to visualise based on feels // overtone singing spectrogram https://www.youtube.com/watch?v=UHTF1-IhuC0

essence of project // pauline oliveros ted talk https://www.youtube.com/watch?v=_QHfOuRrJB8

if only i had more time – same theme // tuning meditation, pauline oliveros – https://www.youtube.com/watch?v=g5bj8sO2-WY

if only i had more time – same theme // einstein on the beach, phillip glass – https://www.youtube.com/watch?v=Ty76wEPL-M4&t=1566

other references // zero and one, Laurie Anderson – Difficult Listening Hourhttps://www.youtube.com/watch?v=6YKcVWVuq4Q

presentation question ideas

• why do you think i’m doing this?
• what visuals / sounds come to mind?
• what’s the difference in seeing this vs. spectrogram?
• what difference would it make if I used ml5 pitch detection and visualized the pitch?
• what do you think I learnt from doing this?
• how did it make you feel?

general questions

• why does pitch detection require machine learning? isn’t it just picking the frequency with highest amplitude?
• is there a way to input sound into browser mic?
• how are the bins related to the frequency? what is the minimum vs. maximum frequency the bins are representing? – it’s a lot less nuanced in the lower frequency sounds (really visible on (C5-B7))
• minimal tracks that are contrasting to three voices?
• how long do we have?
• how to store multiple elements in 1 array?
  answer found – https://www.youtube.com/watch?v=OTNpiLUSiB4&list=PLRqwX-V7Uu6ZmA-d3D0iFIvgrB5_7kB8H&index=15
• is this the correct way to initialize the index for an array?
  let waveL = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
• what makes something only able to run on local server?

project tension

• more functions
• new tracks, tushar’s project, mic
• hard to make one more sketch in one day without them being similar
• make it more like a ‘feeling’ supposed to the technical aspect of the track i.e. too influenced by the fft analysis – hard to be authentic to feeling when something is already defined as a measurable thing
• how to pick the right data to represent something
• fine-tuning – spend 20% of the time fine-tuning and capturing the nuances // like adding bias?
• note screen res: 1920:1080

general tensions

SOUND – “What do you see?”

Opening, Three Voices – two sidedness, imbalance, space, blur, dynamic movements

bouncing ball, sometimes out of sync, starts with 1 – goes up to 3, pendulum, pinned in the middle, afraid to move because it’ll affect the 2 sides, complexed juggling, mirror of 2 things, stuck between 2 sisters (from The Shining) singing out of balance, unexpected, gradient, no sharp lines/edges/shapes, totally organic, moving from left to right (stereo sound), no discernible shape, all gradients, pingpong ball going back and forth, mix of waves, birds flying, Perlin noise waves moving further and further out, creepy ambience, searching, dessert, a cold pond with 4-5 frogs, multiple layers of sound, mice singing, people approaching closely and singing in the ears, things appearing and disappearing, church, 3-4 girls singing and walking randomly – they won’t look directly, just go round and round, hollow forest, not a single visual, journey changes as song progresses

VISUALS – “What do you hear?”

Route 1 – hollow, space, imbalance, appearing and disappearing

breathing with pneumonia, raspy breathing, out of breath, black hole, deep ocean, screaming, subtle gradient, pulling in, imbalance, chilled out, meditative, taoism, heartbeat, tunnel, water slide, swooshing sound, ambient noise (no form), waiting for random colors to arrive or something to pick up the visual but it never arrives, space and time collapsing, wind blow, cold, orbit sound, pure, inside big snow, slow voices

Route 2 – two sidedness, preciseness, vibrations

high pitched waggling, mosquito buzz (not pure sound), pinned to a particular pitch, 4 note chord, needle, precise, muddy low sounds (flickering), metal on metal, sharp, twins from The Shining, gap between doors, vases, buzzing neon, pulsating lines, deep house, bird zapper, frame – something needs to go inside it, electricity, light with bad connection, sunshine reflection

Route 3 – multiple origins of sound, chords, consonance / dissonance, bling

chorus, a lot of chords, dissonance, consonance, strong middle range, a lot of space between the ranges, 3 narrow bands, few notes, sliding between notes (not jumping), holy songs, organs, instruments & voices, lights from stained glass windows, dark room, nostalgia, light on and off, fireflies, nyc subway artist, city lights, fastness, business, tempo, trip-hop, acid-jazz, verizon wireless commercial, ultrasonic sound, underwater,

Further exploration – “what do!?”

• Combine sketches according to feedback (Opening, Three Voices)
• Move on to other songs
• Use ml5 Pitch Detection to analyze song
  • visualize based on notes / harmony

    

• Work on Tushar’s harmony project
  • a person sings a note and the computer creates a song / harmony

Playtest:

• listen to sound
• how do you feel
• what visuals do you see
• composition / color / shape
• show v1 / v2 / v3
• what do you hear

Questions:

• how to store multiple elements in 1 array?
  answer found – https://www.youtube.com/watch?v=OTNpiLUSiB4&list=PLRqwX-V7Uu6ZmA-d3D0iFIvgrB5_7kB8H&index=15
• is this the correct way to initialize index for an array?
  let waveL = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
• what makes something only able to run on local server?

Song reference: https://www.youtube.com/watch?v=EZVsEbodf6o

Song is separated by left and right sounds.

Route 1

https://editor.p5js.org/nt1475@nyu.edu/sketches/SkCKOLJCX

fullscreen: https://editor.p5js.org/nt1475@nyu.edu/full/SkCKOLJCX

Route 2

https://editor.p5js.org/nt1475@nyu.edu/full/BJjobbW0X

fullscreen: https://editor.p5js.org/nt1475@nyu.edu/full/BJjobbW0X

Route 3

https://editor.p5js.org/nt1475@nyu.edu/sketches/SJUH2iZRX

fullscreen: https://editor.p5js.org/nt1475@nyu.edu/full/SJUH2iZRX

Other tests

NOTE TO SELF:

I don’t know a lot about sound and music but they have a lot of influence on me. Last week’s class made me question how sound works, how our brains register different sounds and how they can influence us. After last week’s class on sound I went down a rabbit hole.

First – “Everything You Should Know About Sound”
I feel your vocal cords vibrating

• “The most important kind of human communication happens when our brains send information to other brains through complex patterns of air pressure waves. Have you ever stopped and thought about how incredible that is?” – https://waitbutwhy.com/2016/03/sound.html
  

Second – Musician Explains One Concept in 5 Levels of Difficulty (Harmony)
How is there so much harmony? What are we really hearing?

• 
  https://youtu.be/eRkgK4jfi6M

Third – Simple Waveforms Combined
Look at those weird patterns

• 
  https://www.youtube.com/watch?v=uJDDI9beju4&t=300s

Fourth – Online Tone Generator
Playing different notes that sound unfamiliar

• 
  http://www.szynalski.com/tone-generator/

… and so on.

At this point I know I wanted to work with sound.

Inspiration

• body synth – https://creatability.withgoogle.com/body-synth/
  what if you and your movement become the note?
  
• webcam sequencer – https://jeonghopark.github.io/scanseqjs/
  what is this hidden meaning my face is creating?
  
• slider dance – https://www.youtube.com/watch?v=CMsD3IigG7g&t=646s
  how about switch roles with slides? let the sound turn the slides? and we turn the sound?
  
• self text fill – https://www.youtube.com/watch?v=P_Fx1yq3A8M
  play with lyrics?
  
• petri dish – https://bit.ly/2OrDPQm
  have lyrics bouncing off according to the beat?
  make lyrics explode and rejoin?
  
• bendy grid – https://doseofted.com/project/aesthetic/app.htm?settings=1g54a1aWaFd11a10a176a1t48a2a2aWaBbGF68CFFaFa80a208a2mAesthetic.46.
  this connotes the sense of wavy well? this is similar to what I see in my head.
  
• word synth – https://creatability.withgoogle.com/word-synth/
  try matching words with pitch?
  

Source of materials

• lyrics – https://apiseeds.com/documentation/lyrics
• particle system – https://www.youtube.com/watch?v=vdgiqMkFygc&t
• algorithmic botany – https://www.youtube.com/watch?v=KWoJgHFYWxY&t

Other things I might want to explore

• Algorithmic patterns that exist in nature… might they be related to sound? … What if a value in a pattern is incremented in the same way a note does?
• What if we can reach inside a wave and touch a note? Bend a wave? But make the visual stay… like a harp but with visuals?
• Is there something that is beautifully out of proportion… since our brains just want patterns?
• When is something hypnotic vs. meditative?

A sketch I made with lyrics

https://editor.p5js.org/nt1475@nyu.edu/sketches/ByAgI4J6X

Context?

Who’s it for?

Mostly myself … and non-musicians who enjoy music/sound and want to learn how they work.

How will people experience it? Is it interactive?

Visually, sonically.

Body movements.

Is it practical? No. Is it for fun? Yes. Is it emotional? Maybe. Is it to provoke something? Yes… Question the sounds… Seek for nuances in sounds… Observe it more.

For this week’s assignment on video & sound, I decided to work on sound. I do not know a lot about sound but I’ve always loved music. Learning about what sound actually is and how it actually works was mind opening.

“The most important kind of human communication happens when our brains send information to other brains through complex patterns of air pressure waves. Have you ever stopped and thought about how incredible that is?” – Everything You Should Know About Sound – Tim Urban

https://waitbutwhy.com/2016/03/sound.html

The musical piece I chose to visualize was “Bloodstream” by Tokio Myers. It is a combination of Clair de Lune, Debussy and Bloodstream Ed Sheeran. I didn’t know either of the songs before I heard Tokio Myer’s edition so they both felt like one song to me. But they’re very different in many ways.

My sketch: https://editor.p5js.org/nt1475@nyu.edu/sketches/ByywnRB3X