Storyboarding
I tried to imagine and sketch out a variety of interactions I could map out and what kind of effects they would have on the behaviour of people who were engaging with the project. For this part of the process, I disregarded what kind of technology I would use to map gestures and what kind of visual output it would produce.
Scenario A: Body Position
Starting with something simple: mapping just the x, y, and z coordinates of a person standing somewhere in a limited space, with respect to a predetermined point that I would set as (0, 0, 0) - for the purposes of this sketch, I defined it on the middle of the screen on the ground level.
The x and y coordinates map a person’s position in the space. The z coordinate maps the person’s distance from the ground - so people of different heights, or people bending down or sitting would have different z positions.
I feel like the primary behaviour this would inspire is moving around the space, i.e., changing your x, y, and z positions around the room (last three panels). Another thing that changes might be the speed at which the viewer moves - walking slowly and running may have different effects on how the image on the screen changes.
When two people interact with the installation simultaneously, the effects of doing any of the above will be amplified - what happens when both people are doing the same thing? What happens when they’re performing opposite actions?
Scenario B: Hand Positions
We use our hands for almost every behaviour we perform during the day. It could be interesting to map the different ways hands interact with each other. For this experiment, I imagined tracking nodes at every major joint in a person’s palm and fingers. The artwork on screen would be controlled by four sets of these nodes - each person’s right and left hands - and their distances/positions with respect to each other.
Based on how the artwork changes based on node positions, the experience would ideally encourage people to move their hands around, interacting with each other in a variety of ways as illustrated in the last three panels. I also think levels of familiarity between different people will affect behaviour - strangers probably are not going to thread fingers together, while people who are already friends may skip the handshaking and move into familiar gestures.
Scenario C: Facial Expressions
Facial expressions say a lot about how we experience and interact with the world. Something I imagine would be interesting to track is the position of different facial features - irises, cheeks, nose, corners of the mouth - and how they shift when people change their facial expressions.
This is probably the least mobile setup out of all I’ve imagined - all people need to do is essentially stand in one place facing one direction and make different expressions. It is also the one with the least potential for interaction - while facial expressions are affected by the people around us, they are largely a personal choice and this installation would probably be experienced well enough alone.
This setup would majorly encourage people to exercise their facial muscles by contorting their faces into different expressions - as I speculate in the last three panels above - and observing what it does to the artwork on the screen. Different people with different natural faces will have different effects on the installation even when they’re making identical expressions. If more than one person is interacting with the project together, the combination of both their expressions could lead to a collaborative effort to create more interesting art.
