Awards & Reflection
Musical Plants was awarded as the Most Innovative Project for the subject Designing Novel Interaction 2024, the trophy was awarded by a representative of Melbourne Science Gallery and University of Melbourne's Engineering and IT department.
What I learned from this project is how to think speculatively and pushed myself to go outside of my comfort zone to learn new skills. Specifically, how to integrate hardware and documentation skills which proves to invaluable in later projects.
Musical Plants’s final form is a collection of 3 plants, two Devil Ivys’s and one beetle leaf plant. We tried to set the project to resemble the sketches as much as possible, with a bluetooth speaker connected to act as a the sound output. Additionally, we attached LED’s lights as an additional reactive component that turns on when someone’s touching the plants.
My teammate, Tan was responsible for routing the data from the plants to Arduino and FL studio. Cala and I worked closely together to ensure that the audio was responsive enough to the sensor data coming in.
We did many rounds of testing to make sure that people felt the sound was reactive when they were touching the plants.
Musial Plants works by ultilising the bio electrical signals created from when humans touching the plants, these signals are captured by the capacitive touch sensor that react to human touches and were sent to the microcontroller.
Using an Arduino Leonardo microcontroller, this live data from the plants is routed into the audio synthesis environment FL studio where it is used to control different aspects of a composed sound piece.
Given that we wanted the plants to be touch sensitive, we decided to explore capacitive touch sensing. Capacitive sensing is full of challenges, but after multiple resoldering and testing, we got it to perform in a stable way. Below you can see an early test where the data changes on the screen as I touch the sensor.
Process
In our initial sketches, we imagined different kinds of projects, from interactive mushroom walls to talking plants. We needed to figure our what sensors would gave us a range of data that we could use, and could be somehow influenced by human interaction and intervention, in order to make considered design choices.
Through early testing and prototyping, we refined the concept into a touch and light sensitive system. The project uses capacitive touch sensors integrated with a DAW (Digital Audio Workstation) like FL studio, enabling users to create music through direct interaction with plants. As users touch the leaves or stems, they trigger changes in sound. Each plant is mapped to a specific pitch range, allowing users to compose melodies intuitively through tactile engagement with living elements.
“What if nature could be an interface for controlling sound? If plants could talk, what would they sound like?”
Challenge
The task was to develop a speculative project for exhibition at the Melbourne Design Week 2024, to be showcased at the Science Gallery Melbourne exhibition and aligned with the theme Not Natural. This exhibition explores the growing tension between natural and artificial systems, challenging and blurring the boundary between what is considered natural and what is not.
Our initial concept involved fabricating artificial mushroom walls that would respond to environmental interactions. However, following further discussions and drawing inspiration from prior projects that combine plants and music, we expanded the idea to transform living plants into playable instruments capable of controlling sound.
Musical Plants is an interactive electronic interface that turns plants into playable musical instruments.
Each plant is tuned to a specific pitch range, allowing users to create melodies by simply touching the leaves. The interface connects to a DAW like FL Studio, enabling users to produce real music using a variety of instruments and effects.
This project was showcased at Melbourne Design Week 2024, Musical Plants won the Most Innovative Award at the Cybernetic Futures Showcase.
Role
Concept, Interaction Design, Physical Computing & Hardware
Collaborator
Acalapati Priyatama - Concept, Fabrication
Sakdiphat Tanphiphatari - Programming
Year
March 2024 - May 2024
Musical Plants