The team started from a shared interest in music and accessibility. Our artist used ShapesXR to quickly prototype a kitchen and test the comfort and feel of the experience on Meta Quest.
Developing SpicyTracks: A Musical Kitchen App in Mixed Reality
- Featured
- VR
- Fun&Games
Introduction
What was this project about?
SpicyTracks was created during MIT Reality Hack 2025. Our team designed a mixed-reality app that takes place in a kitchen where everyday items become musical instruments, making music creation more playful, intuitive, and accessible.
My Role
I led the XR development in Unity, implemented interactions such as grab, poke, and gaze-based controls, integrated the 3D models and sounds created by our artists, and added the sound recording system developed by the other developer.
Technical Details
XR Development: Unity, C#.
Timeline
4 days
TL;DR
Problem
Music-making can be difficult to access for people without musical training or with mobility limitations.
- Co-creating music can be even harder when people have different abilities or levels of musical training.
Solution
We created a mixed-reality kitchen where everyday objects become musical instruments.
- Users can grab a frying pan, poke a toaster, or use their eyes to look at items such as glasses and spices to trigger sounds.
- Recording and looping features support collaborative music-making.
- The experience was designed around a pentatonic scale so users could create music freely while still producing harmonious results.
information
Process
I implemented the main kitchen scene in Unity and developed several item interactions, including grabbing, poking, and gaze-based musical controls. I integrated the 3D models and sound assets created by our artist and sound designer, making sure the interactions felt responsive and connected to the kitchen objects. I combined the interactive kitchen scene with the sound recording system developed by the other developer, allowing users to record and layer their musical tracks.
Through multiple tests during the hackathon, we refined the experience to make the interactions more intuitive, playful, and comfortable.
Challenges Faced
Making interactions feel musical
- Making physical interactions such as grabbing, shaking, poking, and looking feel naturally connected to sound generation.
Harmonizing sound and visuals
- The team worked carefully to make the visual style and sound design feel like one cohesive world.
XR build and device issues
- We ran into unexpected device and build issues early in the hackathon, but resolved them quickly with help from mentors.
Impact
Accessible music-making
the project explored how XR can make music creation more intuitive for people without musical training or with different mobility needs.