Find Your Flow

00100dPORTRAIT_00100_BURST20181202203256817_COVER.jpg

‘Find Your Flow’ is a touch sensitive interactive installation that asks you to feel every pixel of its sequin surface and follow its vibration patterns until you ‘Find Your Flow’.
Our physical movement and sense of touch are a large part of our everyday perception, yet we only passively pay attention to them. By tuning into our senses, we become more perceptive of ourselves and our surroundings, and are more able to enjoy the nuances around us.
The installation tracks your movement i.e. your position and speed using a grid of FSR sensors; when you move at the right pace and in the right position for a certain period of time, the vibration stops, and the installation lets you continue on by yourself.
Mechanism: grid of 16 FSR sensors and 16 vibration motors
Material and size: sequin on soft pillow (80 sq.cm. wide, 15cm. high)
Additional elements: small LCD display displaying the status and direction for the installation
Keywords: touch sensitive, vibration, sequin, pixels, slow motion, movement, soft touch

47574832_360409758099698_3945942190349352960_n.png

(credit: Louise)

MVIMG_20181208_000047.jpg

MVIMG_20181208_014320.jpg

(sensor fibonacci)

Screen Shot 2561-12-08 at 02.06.58.png

(experimenting with additional output – sounds from sonic pi)

sound frankenstein

track 1

Untitled presentation (3)_Page_01.jpg

track 2

Untitled presentation (3)_Page_11.jpg

PROCESS

  • pick tracks
    • i selected two minimal sounding tracks (with recommendations from Mimi). why minimal? more clear to how they make me feel i.e. easier to tune in.
      • track 1 – Three Voices by Morton Feldman
      • track 2 – Prayer by Huun Huur Tuu
  • tune in to how i feel
    • this was a lot more difficult than expected. even though i am a non-musician, when trying to ‘listen’ to something, i often internalize it logically i.e. in terms of notes, pitch, rhythm etc. perhaps because there are the visuals associated with those i.e. sheet music, spectrograms, itunes visualizers.
    • picking minimal tracks helped a lot with this – more single-minded
  • sketch out on paper
  • look at data from track
    • fft analysis – analyses the amplitude at different pitches, pick number of bins from 16-1024, pick smoothness / noisiness
      • try to visualize this in different ways to see if there are patterns corresponding to my sketch on paper / how i feel
        Screen Shot 2561-11-16 at 15.33.08.png  screen-shot-2561-11-19-at-10-00-18.png
        Screen Shot 2561-12-02 at 15.56.26.png
      • this is the fabric of all things, spotting the right patterns can lead to very satisfying results
    • get energy – take ranges of bins and averages out the values
      • this is a lazy way … although sometimes this works well, it lacks the nuances which sometimes give insights to the tracks … because it ‘averages’ out the data.
    • amplitude – overall volume
      • since it is an ‘overall’ data, this works well when used to set the ‘tone’ of the drawing, equivalent to a canvas’s quality or primer in a painting.
  • start visualizing (coding)
    • this process is full of wonderful surprises // numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships,
    • time plays an important role in making you feel something (much like in animation) … frame rates, increments, decrements // factor this into numbers and relationships
  • fine-tune visuals
    • fine-tuning takes the most time – changing the transparency by 1% can change the whole drawing. this is a lot of going back and forth between how i feel from listening to the music and what i see
  • know when to end
    • when your ears and eyes feel like they match. i find this process a lot similar to a painting, going too far makes the sketch look ‘overdone’
    • caution: do not get carried away by data – must go back to tuning in process // it is very easy to be lured by the sophisticated visuals (this happens often when headphones are off and i’m just making the sketch ‘look nice’)

NOTE TO SELF

  • the best advice i received was to ‘just put in the hours’ // it is like trying to learn illustrator’s black and white mouse tools (so weird)

CONCLUSION

  • this exploration started because i wanted to hear what musicians were hearing, with a hypothesis that understanding the technical aspects of it and realizing its patterns would help me appreciate and ‘hear’ music more. however, through the exercise i realized it wasn’t about what went in to create the music, rather it’s about listening and realizing what and how exactly a particular sound makes me feel.
  • when listening to music, the feelings i get do not translate to spectrograms, rather some abstract, undefined … stuff
  • these visuals work as a bridge to help me define and express my feelings of a musical piece
  • i hope these visuals stimulate people to ‘listen’ and that they bring people closer towards their feelings (that they might or might not know are there) when they listen to a piece of music.
  • conclusion code: *???* i like arrays because they are like a flexible and well organized box of materials 

PROJECT FILES & WEB EDITOR LINKS

project files – tracks, codes, visual thumbnails, other notes // https://drive.google.com/drive/folders/1XkQERxy_yp-saI_JC0BDk5wt1nVYoFUc

notes from playtesting // https://hellonun.blog/2018/11/23/playtest-sound-frankenstein/

track 1

track 2
PRESENTATION SEQUENCE
  • track 1
    • play track 1 – how do you feel?
    • play visuals for track 1 – can you see what you feel from what you hear?
  • track 2
    • play track 1 – how do you feel?
    • play visuals for track 1 – can you see what you feel from what you hear?

POST-PRESENTATION QUESTIONS FOR CLASS

  • how do you feel?
  • what are the similarities and differences between what you see and what you hear?
  • what if you were to visualise the tracks? what would it be like?

 

FUN LINKS

interesting thing about frequency // wine glass breaking https://www.youtube.com/watch?v=z6oqPB07X3o

everyday sounds we look pass (or hear pass) // sonic branding https://www.youtube.com/watch?v=S_gBMJe9A6Q

it’s about communicating emotions // comment on jacob’s video – https://www.youtube.com/watch?v=eRkgK4jfi6M

what actually happens when you create a sound – the visual is more like this than like notes or other metrics // tom thum – https://www.youtube.com/watch?v=LqdFL0u2HLY

reason to visualise based on feels // feldman site http://www.cnvill.net/mfhome.htm 

reason to visualise based on feels // overtone singing spectrogram https://www.youtube.com/watch?v=UHTF1-IhuC0

essence of project // pauline oliveros ted talk https://www.youtube.com/watch?v=_QHfOuRrJB8

if only i had more time – same theme // tuning meditation, pauline oliveros – https://www.youtube.com/watch?v=g5bj8sO2-WY

if only i had more time – same theme // einstein on the beach, phillip glass – https://www.youtube.com/watch?v=Ty76wEPL-M4&t=1566

other references // zero and one, Laurie Anderson – Difficult Listening Hourhttps://www.youtube.com/watch?v=6YKcVWVuq4Q

 

ICM questions

presentation question ideas

  • why do you think i’m doing this?
  • what visuals / sounds come to mind?
  • what’s the difference in seeing this vs. spectrogram?
  • what difference would it make if I used ml5 pitch detection and visualized the pitch?
  • what do you think I learnt from doing this?
  • how did it make you feel?

general questions

  • why does pitch detection require machine learning? isn’t it just picking the frequency with highest amplitude?
  • is there a way to input sound into browser mic?
  • how are the bins related to the frequency? what is the minimum vs. maximum frequency the bins are representing? – it’s a lot less nuanced in the lower frequency sounds (really visible on (C5-B7))
  • minimal tracks that are contrasting to three voices?
  • how long do we have?
  • how to store multiple elements in 1 array?
    answer found – https://www.youtube.com/watch?v=OTNpiLUSiB4&list=PLRqwX-V7Uu6ZmA-d3D0iFIvgrB5_7kB8H&index=15
  • is this the correct way to initialize the index for an array?
    let waveL = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
  • what makes something only able to run on local server?

project tension

  • more functions
  • new tracks, tushar’s project, mic
  • hard to make one more sketch in one day without them being similar
  • make it more like a ‘feeling’ supposed to the technical aspect of the track i.e. too influenced by the fft analysis – hard to be authentic to feeling when something is already defined as a measurable thing
  • how to pick the right data to represent something
  • fine-tuning – spend 20% of the time fine-tuning and capturing the nuances // like adding bias?
  • note screen res: 1920:1080

general tensions

 

Playtest Sound Frankenstein

SOUND – “What do you see?”

Opening, Three Voices – two sidedness, imbalance, space, blur, dynamic movements

bouncing ball, sometimes out of sync, starts with 1 – goes up to 3, pendulum, pinned in the middle, afraid to move because it’ll affect the 2 sides, complexed juggling, mirror of 2 things, stuck between 2 sisters (from The Shining) singing out of balance, unexpected, gradient, no sharp lines/edges/shapes, totally organic, moving from left to right (stereo sound), no discernible shape, all gradients, pingpong ball going back and forth, mix of waves, birds flying, Perlin noise waves moving further and further out, creepy ambience, searching, dessert, a cold pond with 4-5 frogs, multiple layers of sound, mice singing, people approaching closely and singing in the ears, things appearing and disappearing, church, 3-4 girls singing and walking randomly – they won’t look directly, just go round and round, hollow forest, not a single visual, journey changes as song progresses

VISUALS – “What do you hear?”

Route 1 – hollow, space, imbalance, appearing and disappearing

Screen Shot 2561-11-19 at 09.52.42.png

breathing with pneumonia, raspy breathing, out of breath, black hole, deep ocean, screaming, subtle gradient, pulling in, imbalance, chilled out, meditative, taoism, heartbeat, tunnel, water slide, swooshing sound, ambient noise (no form), waiting for random colors to arrive or something to pick up the visual but it never arrives, space and time collapsing, wind blow, cold, orbit sound, pure, inside big snow, slow voices

Route 2 – two sidedness, preciseness, vibrations

Screen Shot 2561-11-19 at 09.54.36.png

high pitched waggling, mosquito buzz (not pure sound), pinned to a particular pitch, 4 note chord, needle, precise, muddy low sounds (flickering), metal on metal, sharp, twins from The Shining, gap between doors, vases, buzzing neon, pulsating lines, deep house, bird zapper, frame – something needs to go inside it, electricity, light with bad connection, sunshine reflection

Route 3 – multiple origins of sound, chords, consonance / dissonance, bling

Screen Shot 2561-11-20 at 11.13.31.png

chorus, a lot of chords, dissonance, consonance, strong middle range, a lot of space between the ranges, 3 narrow bands, few notes, sliding between notes (not jumping), holy songs, organs, instruments & voices, lights from stained glass windows, dark room, nostalgia, light on and off, fireflies, nyc subway artist, city lights, fastness, business, tempo, trip-hop, acid-jazz, verizon wireless commercial, ultrasonic sound, underwater,

Further exploration – “what do!?”

  • Combine sketches according to feedback (Opening, Three Voices)
  • Move on to other songs
  • Use ml5 Pitch Detection to analyze song
    • visualize based on notes / harmony

      Screen Shot 2561-11-22 at 11.07.35.png

  • Work on Tushar’s harmony project
    • a person sings a note and the computer creates a song / harmony

 

Pcomp playtest and development – “FIND YOUR FLOW”

 

MVIMG_20181115_143147.jpg

Playtesting comments

TOUCH – https://editor.p5js.org/nt1475@nyu.edu/full/H1oND-jT7

MVIMG_20181115_143157.jpg

  • therapeutic
  • paper is rough, feels like papercut
  • want to go back and forth, non linear movement

MOVE – https://editor.p5js.org/nt1475@nyu.edu/full/rkRRIZiTX

MVIMG_20181115_143201.jpg

  • people were more active with it (move fast)
  • need to sense motion as well, not only tilt

BREATHE – https://editor.p5js.org/nt1475@nyu.edu/full/SJs4Lbsa7

MVIMG_20181115_143207.jpg

  • a lot of questions about hygiene
  • is this really a role for tech? or yoga?
  • what is an effective actuator in this case? most things would be distracting

CONNECT – https://editor.p5js.org/nt1475@nyu.edu/full/BJ969bsTm

MVIMG_20181115_143204.jpg

  • do people really sync? we’re biologically different, my slow might not be your slow?

After playtesting the 4 ideas – touch, move, breathe, connect, it was apparent that touch was most effective in slowing people down (given the reasons above as well as the context of it being a semi-public interactive installation).

As a result I developed it further and came up with the idea below.

“FIND YOUR FLOW” – Tai Chi x Sequins

Image result for tai chi

Related image

An interactive sculpture to slow you down.

New Doc 2018-11-19 21.12.39_1.jpg

Schematics to power motors

Screen Shot 2561-11-20 at 09.43.23.png

Transistor array 2400
H bridge chip