Physical Computing

Embodied Interaction

21.04.23 Froject ft Agata Rudnicka

Froject is an interactive experience where you get to experience a small slice of the world as a frog.
We built a 360 experience using unity, touchdesigner, kinect and the igloo 360 room.
The visuals are supposed to represent how a frog views the world based on our current understanding of frog vision.
You can move around the room by bouncing on an exercise ball to simulate a frog hopping.
You can eat fireflies by making a gesture where you bring your hands to your face with your mouth open.
We're handling the body tracking in touchdesigner and sending it to unity via osc.
Unity is being run on the igloo machine thats plugged into the projectors but touchdesigner is being run separately on a laptop plugged in to the kinect.
Tracking on the kinect is great for detecting joint position but getting rotation and gestures reliably is a bigger struggle.
More time would be needed to calibrate the gestures tracking for different people to make it detect more consistently and reliably.
Using the igloo with unity is a bit quirky. We were having frame rate issues that were resolved by running unity in background mode.

06.02.23 Bowow ft Anh & Kaya

We were tasked to come up with a fictional gesture project for embodied interaction. So, we wanted to explore the relationship between man and machine, and the respect shown or lack thereof.
We took our current interactions with laptops, and noticed how demanding we our with them. Everything needs to be instant, they need to be cutting edge, and are easily replaceable if not.
We wanted to shake up that relationship so we picked bowing as our gesture since it is a sign of respect. The idea being that you would have to stand up and bow to the laptop at certain times or during certain interactions.
For example when you start working you would bow before your laptop opened up. It would be the same when you finish work.
And during at times when you start getting frustrated or need to take a break, the laptop would remind you to do so by bowing, so that you have to stop your work, and you would have to bow to start working again. This would be a good time to walk, clear your mind, or grab a coffee since you would have to stand up anyway.

Physical Computing

16.12.22 quiet...

My initial idea was to create a reactive sculpture using muscle wire, in the shape of a blob. With the goal of creating something that clearly doesn't look organic but behaves and moves organically.
Working with muscle wire was challenging, the length and thickness of the wire affects it's resistance and strength so you need to be precise in measuring and cutting to size. Also the wire is super thin so I wasn't able to measure the resistance or current flowing through the wire without crimping it first. Which meant getting the right size of wire required a lot of cutting, crimping, trial and error.
Once I tested everything, I found that the wire wasn't quite strong enough to pull a material and cause it to bend in the blob shape I desired. I think if I had more knowledge of material science, I would have been able to find a material that would have given me the desired effect but I don't so my tests failed. I tried out different shapes and I found that creating a loop makes the wire push out in an interesting way and it might be strong enough to lift up a light fabric.
I laid out the wires in a grid of loops and connected all the electronics. And it was able to lift up a small square of fabric/tissue. At this point it was getting close to the deadline so I had to start finalizing the project so with the help of my friends (thanks Agata and Burak) I made a breadboard and soldered on all the electronics so that I had a standalone disc that contained the whole piece.
The fact that the robot moved in complete silence inspired the way I had it interact with the world. Firstly I found the way that the wires moved when you cycled feeding it current mesmerizing. It looked like it was breathing, so I gave it deep breath and shallow breath movement patterns. I had it default to the deep breath movement and added a sound sensor so that when it detected a loud noise it would switch to the shallow fast breathing until the noise died down. The sound sensor data was pretty noisy (hah) so I ended up using the mic that was built into the arduino nano instead.
Ultimately I found that the movement when the fabric was on top of the mesh to be too subtle so I decided to just leave it exposed. Things didn't go as planned with this project but I do enjoy seeing it breathe so I'll take it.

22.11.22 2 Sensors 1 Cable

I used a pot and a bend sensor to make weird eyes in processing. Arduino Code:

            
          #include 
#include 

int pot;
int bend;
void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);

}

void loop() {
  // put your main code here, to run repeatedly:
  pot = analogRead(A3);
  bend = analogRead(A2);
  Serial.print(pot);
  Serial.print(',');
  Serial.println(bend);
  delay(20);
}

Processing Code:

            
import processing.serial.*;

Serial myPort;  // Create object from Serial class
String str;      // Data received from the serial port
float pot;
float bend;
float c = 10;
float s=10;
void setup()
{
  size(800, 600);
  // Open whatever port is the one you're using.
  // Change the 0 in  Serial.list()[0] to the correct device
  String[] ports = Serial.list();
  printArray(ports);
  String portName = ports[ports.length-1];
  myPort = new Serial(this, portName, 115200);
}

void draw()
{
  
  if ( myPort.available() > 0) {  // If data is available,
    str = myPort.readStringUntil('\n');         // read it and store it in str
    if(str != null){
      String[] splitData = split(str, ",");
      if(splitData.length >= 2)
      {    
        pot = float(splitData[0]);
        bend = float(splitData[1]);
        c = map(pot,500,1023,10,20);
        s = map(bend,200,450,0,4);
      }
    }
    background(255);   
  fill(255);
  int count = (int)c;
  int handle = 4;
  int handleX = 10;
  int eyeWidth = (width/count);
  int eyeHeight = (height/count);
  float openness = -s;
  handle = handle + (int)openness;
  for(int i = 0; i < count+1; i++)
  {
    for(int j = 0; j < count; j++)
    {
      int sizeX = eyeWidth;
      int sizeY = eyeHeight;
      int offset = -((j%2)*eyeWidth/2);
      
      fill(255);
      
      curve(sizeX*i+offset-handleX,       sizeY*(j-handle)+(eyeHeight/2), 
            sizeX*i+offset,               sizeY*j+(eyeHeight/2), 
            sizeX*(i+1)+offset,           sizeY*j+(eyeHeight/2), 
            sizeX*(i+1)+offset+handleX,   sizeY*(j-handle)+(eyeHeight/2));
            
      curve(sizeX*i+offset-handleX,       sizeY*(j+handle)+(eyeHeight/2), 
            sizeX*i+offset,               sizeY*j+(eyeHeight/2), 
            sizeX*(i+1)+offset,           sizeY*j+(eyeHeight/2), 
            sizeX*(i+1)+offset+handleX,   sizeY*(j+handle)+(eyeHeight/2));
            
      fill(0);
      
      circle(sizeX*i+offset+(sizeX/2), sizeY*j+(eyeHeight/2), count);
    }
     
  } 
  } 
}

14.11.22 Idea pitch

08.11.22 Alt+Ctrl

Hug Machine

I chose hug machine, which is a game where you are prompted to hug a mannequin. The game prompts you to hug in a certain way and based on whether you do the 'right' hug it marks you as right or wrong. The mannequin itself is the controller and can tell where you are hugging it based on what I'd assume is some kind of pressure sensor or button.

I chose this controller because I think the concept of using something vaguely human shaped as a controller fascinating. And the focus seems to be on human touch which is something I'd like to explore.

For my sensor/game, I wanted to use the magnet sensors and a puck shaped magnet. And you flick the magnet towards the sensor which is embedded into a tiny goal. Based on the speed and position it controls the swing for a game called cursed to golf. The position of the swing is based on the position of the magnet and the power is based on the speed.

03.11.22 LED Double Trouble

The task at hand was to create a circuit where you repeatedly pressed a button to turn in two leds in a cyclical sequence; both leds off, one led on, both leds on.

The leds needed to be connected to digital pins in OUTPUT mode so that they could be turned HIGH or LOW from the arduino. And had to pass through a resistor to reduce the current.

The button needed to be connected to the arduino digital in INPUT mode and needed to be connected to a 10k ohm pulldown/pullup resistor.

            
          int led1 = 6;
          int led2 = 7;
          int button = 3;

          int bVal = 0;
          int counter = 0;
          bool pressed = false;
          bool wasPressed = false;

          void setup() {
            // put your setup code here, to run once:
            pinMode(led1, OUTPUT);
            pinMode(led2, OUTPUT);
            pinMode(button, INPUT);

            Serial.begin(9600);
          }

          void loop() {
            // put your main code here, to run repeatedly:
            bVal = digitalRead(button);
            Serial.println(bVal);
            if(bVal > 0)
            {
              pressed = true;
            }
            else
            {
              pressed = false;
            }
            if(wasPressed == true && pressed != wasPressed)
            {
              counter++;
              if(counter > 2)
                counter = 0;
              switch(counter)
              {
                case 0:
                digitalWrite(led1, LOW);
                digitalWrite(led2, LOW);
                break;
                case 1:
                digitalWrite(led1, HIGH);
                break;
                case 2:
                digitalWrite(led2, HIGH);
                break;
              }
            }
            wasPressed = pressed;
            delay(10);
          }