First Year Reflection

Hello! I’ve known I wanted to be a designer since I was a little kid, although the exact field has varied over the years. I’ve been exploring the field of design, in related interdisciplinary fields, for almost ten years. For the past year in DT, I’ve been exploring how technology as a tool can aid my design process, and how it can be used to express myself. I have been building my knowledge base by trying different things I can do with design and technology.

Embodied Impressions is a project I worked on in Major Studio 2 with Jon Thirkield through most of the spring semester. I wanted to build something users could interact with, but also something they would engage with in a way that they would feel personally connected to the outputs of their interaction. The goal was to outline a sense of creation — physical, mental, and embodied — that only existed when a user was interacting with the project, to emphasise the fact that creation lies more in the act of engaging in it than producing a polished output.

The system works through a camera setup, where an Azure Kinect takes in image input and body tracking data, and then maps positions of different body parts using pose detection in ml5.js. The system then creates visual and musical outputs based on the properties of the tracked body parts using p5.js and tone.js.

A full video and explanation here. A complete process documentation journal can be found at this link, and my final presentation for this project is here.

I have never been musically inclined, so working with music in this project required some serious research into music theory and practices. For my Creative Practice Seminar Project Paper, I ended up writing entirely about music, its development, history, and practical use. However, experimentation with music in the first place was inspired by the week we covered Computational Music in one of the most productive electives I have taken so far: Computational Form by Justin Bakse. This class didn’t assign specific projects all semester, and instead required regular sketching using code and procedural concepts every week. I feel like this class was invaluable in demonstrating the power of making, and allowed me to become comfortable with and confident in trying new technologies on a weekly basis. 

I only started coding last semester, in Fall 2022’s Critical Computation. Taking this class helped me not only strengthen my understanding of computational concepts, but also develop confidence in learning new libraries or languages quickly, and equipped me with a thought and making process to grasp new technologies faster. All 60 sketches I did for this class over the semester can be found here.

CYAN: Create Your Album Now is another project I worked on for most of this past semester, in a Collab elective called Machine Learning & the City with Sven Travis and Christopher Kirwan. The class was an exploration into the potential of different ML platforms and how they can be used. Continuing the theme of music, Nidhi Nagabhushan and I used this time to explore different aspects of music and how AI could create/augment them.

We identified different artists from different genres and different time periods contextualised in different neighbourhoods in New York City, and used these parameters to come up with a prompt in the format “write a song about [location] in new york city in the genre [genre] set in the [time period] in the style of [artist]” that we fed into various ML platforms to create music — Riffusion for instrumental music, ChatGPT for lyrics, and Midjourney for album cover images. We then surveyed people in these areas of the city to learn if they could identify AI music and what they thought of it. We ended up creating a playlist mockup of all of these songs created using AI. 

Through the course of this project, we came up with interesting observations and insights into the biases and limits of these platforms. Recently, Parsons’ re:D design magazine interviewed us to feature this project in an article about AI in their next issue.

Link to final presentation here. The playlist can be found here.

A key piece of advice that allowed me to be be more exploratory in my work, without worrying about failure — especially when working with music, a field I have little experience in — is when about midway through the semester, some of my faculty across a couple of classes stressed that the work we do can just be an exploration or a series of experiments, and doesn’t need to be a fully polished project. I feel like this took away the pressure of needing to come up with a complete project that had little room for improvement or error, and allowed me to try a host of ideas, some of which worked and some didn’t, but all of which ultimately led to conceptually stronger work that has potential to develop further.

One thing I do wish I had done this past semester, especially as it pertains to my Major Studio project, is start making earlier. I did spend a few weeks before the midterm trying out a variety of technologies I could use to develop my motion tracking system, most of which didn’t work — but I didn’t start working on outputs until 5-in-5, the week before midterm break — and the first time I put everything together was the Prototype 1 presentation, a couple of weeks after the midterm. Many avenues for improvement and ideas for next steps came up after I started making — and in a project like mine, especially after I started user testing. I feel like starting to make even rudimentary prototypes a few weeks earlier in the semester would have helped immensely in building a better, more well rounded, more user responsive project.

Over the course of the summer, I want to continue building my knowledge base, especially when it comes to computational concepts. I want to continue working on some of the projects I developed this semester — particularly Embodied Impressions, as well as a couple of narrative projects I developed for a different elective. As I focus on learning more technological tools and their potential through the next year in DT, I want to continue working on projects that involve user engagement, and explore how I can best express my intentions through my design process and how people respond to it.