Most innovative project
Melbourne Design Week 2024
Musical Plants
An interactive interface that turns living plants into playable instruments — touch a leaf, and it sings.
Role
Concept, Interaction Design, Physical Computing & Hardware
Team
Acalapati Priyatama — Concept, Fabrication
Sakdiphat Tanphiphatari — Programming
Timeline
March – May 2024
10 weeks
Tools
Arduino Leonardo, capacitive sensors, FL Studio, Fusion 360
Each plant is tuned to its own pitch range. Users compose melodies just by touching the leaves.
The interface connects to a DAW (digital audio workstation) such as FL Studio, so people can make real music with a range of instruments and effects, without any musical training.
It was shown at Melbourne Design Week 2024 and won the Most Innovative Award at the Cybernetic Futures Showcase.
Demo — each plant plays a different range of notes in different octaves
01
Challenge
The brief: develop a speculative project for Melbourne Design Week 2024, shown at Science Gallery Melbourne under the theme Not Natural. The exhibition looks at the growing tension between natural and artificial systems, and blurs the line between what counts as natural and what doesn’t.
Constraint
Must fit a public gallery: people walk up with no instructions.
Constraint
Must respond to people touching it in a way they can feel right away.
Constraint
Built in 10 weeks, from off-the-shelf sensors and a microcontroller.
“What if nature could be an interface for controlling sound? If plants could talk, what would they sound like?”
02
Ideation
Our first sketches covered very different ideas, from interactive mushroom walls to talking plants. To make each design choice, we had to find out which sensors gave us usable data that people could change by touching or moving near them.
The first concept was a wall of artificial mushrooms that responded to the environment. After more discussion, and inspired by earlier projects that combined plants and music, we moved to living plants played as instruments. A real plant made the “Not Natural” tension easier to feel.
Concept A — Mushroom wall
Dropped
Concept B — Musical plants
Selected
03
Sensing touch
Early testing and prototyping turned the concept into a touch- and light-sensitive system. We wanted the plants themselves to feel touch, so we tried capacitive sensing.
Capacitive sensing is hard to get right. After several rounds of resoldering and testing, it worked reliably. The early test below shows the readings changing on screen as the sensor is touched.
Sensor data reacting to touch in real time
04
How it works
When a person touches a plant, it creates a bioelectrical signal. A capacitive touch sensor picks this up and sends it to an Arduino Leonardo.
The Arduino sends this live data to FL Studio, where it controls different parts of a composed piece of music. Each plant is mapped to its own pitch range.
01
Touch
A visitor touches a leaf or stem.
02
Sense
A capacitive sensor picks up the bioelectrical signal.
03
Process
The Arduino Leonardo reads and sends the live data.
04
Synthesise
FL Studio maps the data to pitch and effects.
05
Respond
Sound plays from the speaker and the LEDs light up.
Plant 1
Devil’s Ivy
Low pitch
Plant 2
Betel leaf
Medium pitch
Plant 3
Devil’s Ivy
High pitch
05
Sound design & testing
My teammate Tan routed the data from the plants through the Arduino to FL Studio. Cala and I worked closely to tune the audio so it kept up with the incoming sensor data.
We tested many times with real people until they felt the sound react the moment they touched a plant. Latency was the main usability measure: any delay broke the feeling that the plant was “playing.”
User testing at the showcase
FL Studio · FLEX “Music Box” preset, used to shape the plants’ voice
06
Final form
The final piece is three plants: two Devil’s Ivy and one betel leaf. We kept the setup as close to the sketches as possible, with a Bluetooth speaker for sound output.
LEDs on each pot light up while someone is touching the plant. This gives visual feedback and shows that the plant is “listening.”
3D-printed pot
A faceted pot with a label holder, modelled in Fusion 360.
Reactive LEDs
Light up on touch to show the plant has responded.
Bluetooth speaker
A wireless speaker keeps the table uncluttered and the sound close to the plants.
Pitch labels
Handwritten notes tell visitors what each plant plays, so no instructions are needed.
07
Outcome & reflection
Award
Most Innovative Project — Designing Novel Interaction 2024
Presented by a representative of Science Gallery Melbourne and the University of Melbourne’s Engineering and IT department.
This project taught me to think speculatively and pushed me past my comfort zone into new skills. Integrating hardware and documenting the work well both proved invaluable in later projects.
Speculative thinking
Starting from “what if”, then working back to something people can use.
Hardware integration
Sensors, soldering and microcontrollers as part of the design material.
Feel over features
Fast response mattered more than how complex the sound was.
Full documentation
See the full visual report
The original long-form report, with every sketch, test and build photo.
View visual report ↗