Find Your Flow – Final Project

‘Find Your Flow’ is a touch sensitive interactive installation that asks you to feel every pixel of its sequin surface and follow its vibration patterns until you ‘Find Your Flow’.
Our physical movement and sense of touch are a large part of our everyday perception, yet we only passively pay attention to them. By tuning into our senses, we become more perceptive of ourselves and our surroundings, and are more able to enjoy the nuances around us.

This project has come a very long way! I started off experimenting with various interactions to ‘slow people down’. From breathing, movement, touching/feeling, human connection. Below is the documentation of the journey!

Phase I – link to ideation https://hellonun.blog/2018/11/08/final-project-week-1-2/

Phase II – link to playtest https://hellonun.blog/2018/11/20/pcomp-playtest-and-development-find-your-flow/

Phase III – link to winter show (concept) https://hellonun.blog/2018/12/05/find-your-flow/

Find Your Flow live on Coding Train – https://www.youtube.com/watch?v=9B7zU-yyLAk&t=7340s

Screen Shot 2561-12-18 at 23.45.25  Screen Shot 2561-12-18 at 23.45.20


 

Find Your Flow Project Image

00100dPORTRAIT_00100_BURST20181202203256817_COVER.jpg

Elaboration on last stage:

Once I decided on the design for find your flow, I started experimenting with the physical computing aspect of it.

Summary of process

  • design interaction and fabrication
    • touch to activate?
    • how much time to delay so that people would know to slow down?
    • how sensitive does the touch have to be?
    • does it only vibrate when they touch?
    • how will they know when to stop?
  • determine parts
  • what sensor to use
    • in order to create a touch sensitive surface, I experimented with the following sensors
      • Piezo (vibration sensor) – I was recommended by a few people to use this as it was very sensitive. However, because the actuator was a vibration motor, this did not work out no matter how I placed it.
        https://itp.nyu.edu/classes/tangible-interaction/2017/02/25/piezo-sensor/
      • Capacitive (valostat and coppertape) – Although this provided some feedback, it was not sensitive enough. It required me to apply a lot of pressure which did not suit the interaction I was after.

        MVIMG_20181127_122702.jpg

      • FSR circular – This worked very well on its own. However, the surface was very small. Also, when covered with sequin, this was not sensitive enough.

        MVIMG_20181127_122602.jpg

      • FSR squares – THE PERFECT SENSORS! the square FSR’s were perfect! Size: perfect size to make a grid / Touch: perfect under the sequin + motors.
  • how to layer the various components – I ended up putting the vibration motors on top of the fsr sensors as it was giving the most accurate values as well as most effective sense of vibration

    IMG_20181206_031143.jpg

    MVIMG_20181208_144954.jpg

    MVIMG_20181212_042916.jpg
     

     

  • Solder all sensors and motors! The vibration motors came with very thin wires. Not only these were difficult to strip, they broke very easily. After soldering them (use flux!) I used heat shrink and electric tape to make them more firm.
    NOTE: if working with soft fabric, we should use flexible thin wires! I made a mistake in using hard wires thinking they’re more firm but they end up being troublesome because they’re very difficult to bend – also, they create a rough surface.
  • Board layoutScreen Shot 2561-12-18 at 23.05.14.png
  • Code
    • be careful of which pin is which
    • serial print to test but delete the serial for responsiveness for the real code
    • here is the code (repeat for other pins)
      Screen Shot 2561-12-18 at 23.12.17.png

In-class presentation: this did not work! This was sad! My project failed to come to life during the class’s final presentation. However, this forced me to come up with a plan, if all things fail (if the sensors don’t work), which is to create random vibrations.

Key learning

  • ALWAYS SODER AND USE PC BOARDS!!! This became a huge problem as my wires got extremely complicated. I believe there were some signal errors due to the amount of wires in such a condensed space. Sensors would randomly turn on and off even though everything was plugged in the same way – I would have to restart the system by plugging and unplugging the arduino. Due to the limited time, I couldn’t rewire and solder everything. However, I tried to be as neat as possible and taped all the jumper cables together to create a firm grip.
  • Takeout from Winter Show:
    • about 70% of people understand and had the designed interaction with the piece (touch and run their hands through it very slowly). others try to listen to it, draw on it or squeeze it.
       

       

    • I find that kids are a lot more sensitive to touch than adults. they would very slowly touch and be amazed by the vibrations and sequins.
      MVIMG_20181216_141658.jpg

      MVIMG_20181216_150459.jpg

    • some comments
      • pregnant whale
      • qr code generator
      • purring cat
  • I am very pleased with how the project turned out. It was surprisingly effective in terms of getting people to slow down and pay more attention to their sense of touch, feel very sequin on the fabric and stay tuned to its vibration.
    • why does it work? I believe it is because it is on that thin line of having enough feedback to keep you engaged but not too much to determine your reaction
      • the tactile feel / vibration
        • this gives a a feeling that you are reacting to something in a very physical way. much like dipping your hand in a sack of rice, running your hand through grass or feeling a pattern on a surface.
      • the vibration
        • this gives a feeling that something is reacting to you through touch which is different from most technology. according to the users, this feels more ‘real’ and more ‘alive’ in relatively other technologies.
      • the visual feedback
        • because people are often distracted by visuals, this keeps them engaged with the piece. having something for the eyes to focus on.

To be added: video from winter show.


Others:

MVIMG_20181208_000047.jpg

(testing the multiplexers)

MVIMG_20181208_014320.jpg

(sensor fibonacci)

Screen Shot 2561-12-08 at 02.06.58.png

(experimenting with additional output – sounds from sonic pi)

Float House – Unreal Engine

Float House is my last project of the Winter 2018 semester (post-winter show project), hence the calmness! I started this project roughly 2 weeks ago when we were introduced to Unreal engine. It has been challenging navigating in Unreal! Even though I have experience with Cinema4D, Unreal has its own rules and logic (elaborated below).

Work process

  • Fuse – we created the character appearance on Fuse
  • Mixamo – we then uploaded the charater to Mixamo in order to rig (put in joints and bones), as well as to explore various animations
  • Unreal Engine
    • ENVIRONMENT – create the environment / world for our animation
    • T-POSE – import character from mixamo in t-pose format (this is only available as an option of some animations – it prompts you to select the option on mixamo)
    • SCENE (SEQUENCE) – once the t-pose character is imported, we must create a ‘sequence’ in order to begin the animation – this works like a timeline where keyframes can be added on various components
    • ANIMATE – import animations from mixamo and add them on top of the t-pose in order to move the characters
    • CAMERA+EXPORT – once the keyframes are done, add a camera and see the video play from camera view – notice the lighting and focus. Once the camera is adjusted, export the sequence in a video format. Remark: adjust the resolution!

      Screen Shot 2561-12-18 at 22.11.04

      Screen Shot 2561-12-18 at 22.10.08

      Screen Shot 2561-12-18 at 22.10.34

      Screen Shot 2561-12-18 at 22.10.25

    • for future reference: unreal tutorial
      https://www.youtube.com/playlist?list=PLoJ1QVW2Xa7QYlCR5xO9EvFOCRrCPMEko
    • class screen caps
      https://drive.google.com/drive/u/3/folders/1bGVkkfzctGXauHC6-WQQchvHh2zd6T8p
  • Premiere Pro
    • import all the sequences from Unreal engine and fine tune / edit on premiere, add sounds, title etc.

Takeouts:

  • I very much enjoy creating a world on Unreal engine. The amount of detail on texture, shapes, animation, lighting etc. allows another dimension of storytelling e.g. feels calming, modern, cold, high-tech, rough
  • The characters and animations are limited to the presets unless we build our own models and rig our own characters (apparently a very long and complicated process)
  • Overall this process made me appreciate 3D animations more!

Find Your Flow – Winter Show

00100dPORTRAIT_00100_BURST20181202203256817_COVER.jpg

‘Find Your Flow’ is a touch sensitive interactive installation that asks you to feel every pixel of its sequin surface and follow its vibration patterns until you ‘Find Your Flow’.
Our physical movement and sense of touch are a large part of our everyday perception, yet we only passively pay attention to them. By tuning into our senses, we become more perceptive of ourselves and our surroundings, and are more able to enjoy the nuances around us.
The installation tracks your movement i.e. your position and speed using a grid of FSR sensors; when you move at the right pace and in the right position for a certain period of time, the vibration stops, and the installation lets you continue on by yourself.
Mechanism: grid of 16 FSR sensors and 16 vibration motors
Material and size: sequin on soft pillow (80 sq.cm. wide, 15cm. high)
Additional elements: small LCD display displaying the status and direction for the installation
Keywords: touch sensitive, vibration, sequin, pixels, slow motion, movement, soft touch

Contact:

nt1475@nyu.edu

9177941426

sound frankenstein

track 1

Untitled presentation (3)_Page_01.jpg

track 2

Untitled presentation (3)_Page_11.jpg

PROCESS

  • pick tracks
    • i selected two minimal sounding tracks (with recommendations from Mimi). why minimal? more clear to how they make me feel i.e. easier to tune in.
      • track 1 – Three Voices by Morton Feldman
      • track 2 – Prayer by Huun Huur Tuu
  • tune in to how i feel
    • this was a lot more difficult than expected. even though i am a non-musician, when trying to ‘listen’ to something, i often internalize it logically i.e. in terms of notes, pitch, rhythm etc. perhaps because there are the visuals associated with those i.e. sheet music, spectrograms, itunes visualizers.
    • picking minimal tracks helped a lot with this – more single-minded
  • sketch out on paper
  • look at data from track
    • fft analysis – analyses the amplitude at different pitches, pick number of bins from 16-1024, pick smoothness / noisiness
      • try to visualize this in different ways to see if there are patterns corresponding to my sketch on paper / how i feel
        Screen Shot 2561-11-16 at 15.33.08.png  screen-shot-2561-11-19-at-10-00-18.png
        Screen Shot 2561-12-02 at 15.56.26.png
      • this is the fabric of all things, spotting the right patterns can lead to very satisfying results
    • get energy – take ranges of bins and averages out the values
      • this is a lazy way … although sometimes this works well, it lacks the nuances which sometimes give insights to the tracks … because it ‘averages’ out the data.
    • amplitude – overall volume
      • since it is an ‘overall’ data, this works well when used to set the ‘tone’ of the drawing, equivalent to a canvas’s quality or primer in a painting.
  • start visualizing (coding)
    • this process is full of wonderful surprises // numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships, numbers, relationships,
    • time plays an important role in making you feel something (much like in animation) … frame rates, increments, decrements // factor this into numbers and relationships
  • fine-tune visuals
    • fine-tuning takes the most time – changing the transparency by 1% can change the whole drawing. this is a lot of going back and forth between how i feel from listening to the music and what i see
  • know when to end
    • when your ears and eyes feel like they match. i find this process a lot similar to a painting, going too far makes the sketch look ‘overdone’
    • caution: do not get carried away by data – must go back to tuning in process // it is very easy to be lured by the sophisticated visuals (this happens often when headphones are off and i’m just making the sketch ‘look nice’)

NOTE TO SELF

  • the best advice i received was to ‘just put in the hours’ // it is like trying to learn illustrator’s black and white mouse tools (so weird)

CONCLUSION

  • this exploration started because i wanted to hear what musicians were hearing, with a hypothesis that understanding the technical aspects of it and realizing its patterns would help me appreciate and ‘hear’ music more. however, through the exercise i realized it wasn’t about what went in to create the music, rather it’s about listening and realizing what and how exactly a particular sound makes me feel.
  • when listening to music, the feelings i get do not translate to spectrograms, rather some abstract, undefined … stuff
  • these visuals work as a bridge to help me define and express my feelings of a musical piece
  • i hope these visuals stimulate people to ‘listen’ and that they bring people closer towards their feelings (that they might or might not know are there) when they listen to a piece of music.
  • conclusion code: *???* i like arrays because they are like a flexible and well organized box of materials 

PROJECT FILES & WEB EDITOR LINKS

project files – tracks, codes, visual thumbnails, other notes // https://drive.google.com/drive/folders/1XkQERxy_yp-saI_JC0BDk5wt1nVYoFUc

notes from playtesting // https://hellonun.blog/2018/11/23/playtest-sound-frankenstein/

track 1

track 2
PRESENTATION SEQUENCE
  • track 1
    • play track 1 – how do you feel?
    • play visuals for track 1 – can you see what you feel from what you hear?
  • track 2
    • play track 1 – how do you feel?
    • play visuals for track 1 – can you see what you feel from what you hear?

POST-PRESENTATION QUESTIONS FOR CLASS

  • how do you feel?
  • what are the similarities and differences between what you see and what you hear?
  • what if you were to visualise the tracks? what would it be like?

 

FUN LINKS

interesting thing about frequency // wine glass breaking https://www.youtube.com/watch?v=z6oqPB07X3o

everyday sounds we look pass (or hear pass) // sonic branding https://www.youtube.com/watch?v=S_gBMJe9A6Q

it’s about communicating emotions // comment on jacob’s video – https://www.youtube.com/watch?v=eRkgK4jfi6M

what actually happens when you create a sound – the visual is more like this than like notes or other metrics // tom thum – https://www.youtube.com/watch?v=LqdFL0u2HLY

reason to visualise based on feels // feldman site http://www.cnvill.net/mfhome.htm 

reason to visualise based on feels // overtone singing spectrogram https://www.youtube.com/watch?v=UHTF1-IhuC0

essence of project // pauline oliveros ted talk https://www.youtube.com/watch?v=_QHfOuRrJB8

if only i had more time – same theme // tuning meditation, pauline oliveros – https://www.youtube.com/watch?v=g5bj8sO2-WY

if only i had more time – same theme // einstein on the beach, phillip glass – https://www.youtube.com/watch?v=Ty76wEPL-M4&t=1566

other references // zero and one, Laurie Anderson – Difficult Listening Hourhttps://www.youtube.com/watch?v=6YKcVWVuq4Q

 

ICM questions

presentation question ideas

  • why do you think i’m doing this?
  • what visuals / sounds come to mind?
  • what’s the difference in seeing this vs. spectrogram?
  • what difference would it make if I used ml5 pitch detection and visualized the pitch?
  • what do you think I learnt from doing this?
  • how did it make you feel?

general questions

  • why does pitch detection require machine learning? isn’t it just picking the frequency with highest amplitude?
  • is there a way to input sound into browser mic?
  • how are the bins related to the frequency? what is the minimum vs. maximum frequency the bins are representing? – it’s a lot less nuanced in the lower frequency sounds (really visible on (C5-B7))
  • minimal tracks that are contrasting to three voices?
  • how long do we have?
  • how to store multiple elements in 1 array?
    answer found – https://www.youtube.com/watch?v=OTNpiLUSiB4&list=PLRqwX-V7Uu6ZmA-d3D0iFIvgrB5_7kB8H&index=15
  • is this the correct way to initialize the index for an array?
    let waveL = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
  • what makes something only able to run on local server?

project tension

  • more functions
  • new tracks, tushar’s project, mic
  • hard to make one more sketch in one day without them being similar
  • make it more like a ‘feeling’ supposed to the technical aspect of the track i.e. too influenced by the fft analysis – hard to be authentic to feeling when something is already defined as a measurable thing
  • how to pick the right data to represent something
  • fine-tuning – spend 20% of the time fine-tuning and capturing the nuances // like adding bias?
  • note screen res: 1920:1080

general tensions

 

ideas for one day